class: center, middle, inverse, title-slide # <span style="font-size:48pt;">Regularized Regression</span> ## .big[⛰ 🤠 🕸️ ] ### Applied Machine Learning in R </br> Pittsburgh Summer Methodology Series ### Lecture 3-A     July 21, 2021 --- class: inverse, center, middle # Overview <style type="text/css"> .onecol { font-size: 26px; } .twocol { font-size: 24px; } .remark-code { border: 1px solid grey; } a { background-color: lightblue; } .remark-inline-code { background-color: white; } </style> --- ## Lecture Topics .pull-left[ **Linear Regression Review** - Ordinary least squares - Minimizing sum-of-squared-errors - Limitations of OLS regression **Introduction to Regularization** - Why regularize? - Bias-variance tradeoff - Coefficient paths - Feature selection - Tuning hyperparameters ] .pull-right[ **Ridge** - `\(L_2\)` penalty - Parameter shrinkage towards zero **Lasso** - `\(L_1\)` penalty - Some parameters actually go to zero **Elastic Net** - Combining penalty terms - `\(\lambda\)` and `\(\alpha\)` tuning parameters ] --- class: inverse, center, middle # Linear Regression Review --- ## Linear Regression Linear regression and closely related models (ridge, lasso, elastic net, etc.) can all be written in the form: `$$y_i = b_0 + b_1x_{i1} + b_2x_{i2} + ... + b_Px_{iP} + e_i$$` where - `\(y_i\)`: value of the response for the `\(i\)`th observation - `\(b_0\)`: estimated intercept - `\(b_j\)`: estimated coefficient for the `\(j\)`th predictor - `\(x_{ij}\)`: value of the `\(j\)`th predictor for the `\(i\)`th observation - `\(e_i\)`: random error unexplained by the model for the `\(i\)`th observation -- Such models are  because each parameter (e.g., `\(b_1\)`, `\(b_2\)`, etc.) appears only with a power of 1 and is not multipled or divided by any other parameter. Nonlinear *variables* (e.g., `\(x_1^2\)`) can still be included in linear regression as long as the *parameters* remain linear. --- ## Ordinary Least Squares Regression In ordinary least squares regression, the parameters (e.g., `\(b_1\)`, `\(b_2\)`, etc.) are estimated to . </br> <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/complexity-1.png" width="90%" /> --- ##Ordinary Least Squares Regression Specifically, parameters in OLS regression are chosen to minimize the sum-of-squared errors (SSE): `$$SSE = \sum\limits_{i = 1}^n(y_i - \hat{y_i})^2$$` where - `\(n\)`: number of observations - `\(y_i\)`: `\(i\)`th observed response - `\(\hat{y_i}\)`: `\(i\)`th predicted response <br/> -- However, any dataset is influenced by both the underlying data-generating mechanisms *and* , which by definition is not shared with other samples drawn from the same population. --- ##Ordinary Least Squares Regression <img src="data:image/png;base64,#overfit.jpg" width="60%" style="display: block; margin: auto;" /> --- ##Ordinary Least Squares Regression While OLS regression is readily interpretable and easy to compute, there are several noteworthy limitations: - Risk of  -  parameter estimates - Poor  in new datasets - Sensitive to  (adjusting parameter estimates to better accommodate outlier observations with large residuals in order to minimize SSE) - Cannot handle datasets with high  - Cannot handle datasets with  - May not adequately capture  relationships<sup>1<sup> <br/> Regularization addresses many (but not all) of these problems. .footnote[ [1] This remains a problem with regularized regression models, but see methods for adding nonlinearity to linear models in lecture 2-A (Feature Eningeering). ] --- class: inverse, center, middle # Regularization --- ##What is regularization, and why regularize? .left-column[ <br /> <img src="data:image/png;base64,#overfit_graph.png" width="100%" /> ] .right-column[ **Reduce Overfitting** OLS regression models overfit the data and inflate parameter estimates. Regularized regression adds an additional  to the error function, which constrains parameter estimates and penalizes model complexity. Compared to OLS regression models, regularized regression models have a  but . In essense, we make our model less sensitive to the data in the training set in order to achieve higher accuracy in the test set. ] --- ##What is regularization, and why regularize? .left-column[ <br /> <img src="data:image/png;base64,#feature_selection.png" width="100%" /> ] .right-column[ **Feature Selection** We are often interested in finding a subset of 'good' predictors. However, there are are many problems¹ with the traditional approach of stepwise regression models. One benefit of regularization is that it shrinks parameter estimates towards zero. In some cases, such as lasso regression, some parameters are . Thus, regularization models can simultaneously reduce overfitting and perform feature selection. ] .footnote[ [1] see Harrell (2015), section 4.3, for details and explanation: https://link.springer.com/book/10.1007/978-3-319-19425-7 ] --- ## Comprehension check .pull-left[ ### Question 1 **Determine which of the following models are nonlinear.** a) `\(y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \epsilon\)` b) `\(y = \beta_0 + \beta_1^2x_1 + \beta_2x_2 + \epsilon\)` c) `\(y = \beta_0 + \beta_1x_1^2 + \beta_2\sin(x_1x_2)+ \epsilon\)` d) `\(y = \beta_0 + e^{\beta_1x_1} + \beta_2x_2 + \epsilon\)` ] .pull-right[ ### Question 2 **Which is not a benefit of regularized compared to nonregularized models?** a) Feature selection b) Improves out-of-sample prediction c) Overcomes measurement errors d) Limits overfitting ] --- class: inverse, center, middle # Ridge Regression --- ## Ridge Regression Recall the SSE error function for OLS regression: `$$SSE = \sum\limits_{i = 1}^n(y_i - \hat{y_i})^2$$` -- Ridge regression adds a penalty term to this function known as the `\(L_2\)` penalty: `$$SSE_{L2} = \sum\limits_{i = 1}^n(y_i - \hat{y_i})^2 + \lambda \sum\limits_{j = 1}^P \beta_j^2$$` where - `\(n\)`: number of observations - `\(y_1\)`: `\(i\)`th observed response - `\(\hat{y_i}\)`: `\(i\)`th predicted response - `\(P\)`: number of parameters - `\(\beta_j\)`: the `\(j\)`th parameter --- ## Ridge Regression .left-column[ <br /> <img src="data:image/png;base64,#ridge.png" width="100%" /> ] .right-column[ Whereas OLS regression minimizes the difference between the observed and predicted data, ridge regression has an additional goal of  `\(\color{darkred}\lambda\)` . Thus, parameter estimates in ridge regression are only able to become large if there is a proportional reduction in `\(SSE_{L2}\)`. The `\(\lambda\)` parameter is a  controlling the degree of regularization that ranges from [0, inf]. Higher values of `\(\lambda\)` will shrink parameters closer to zero and At higher values of `\(\lambda\)`, parameters will be shrunk closer to zero. We can find the 'best' value of `\(\lambda\)` through cross-validation hyperparameter tuning. ] --- ## Hyperparameter Tuning <img src="data:image/png;base64,#hyperparameter_tuning1.png" width="90%" /> --- ## Hyperparameter Tuning <img src="data:image/png;base64,#hyperparameter_tuning2.png" width="90%" /> --- ## Dataset Today, we'll work with a simulated dataset of 500 'people' examining predictors of eating disorder severity¹. Simulated variables include theoretically *related* and theoretically *unrelated* to eating disorders severity: -- .pull-left[ - Emotion regulation - Depression - Impulsivity - Self-criticism - Anxiety - Race (categorical) - Age - Family psychiatric history - Prior psychiatric treatment - Length of time being in treatment - Perfectionism ] -- .pull-right[ - Temperature that day - Average rainfall over past month - Number of siblings - Number of cellphones owned - Time spent reading the news - Time spent watching TV - Number of pets owned ] .footnote[ [1] Note: Slides will offer a regression example and live coding will offer a classification example in a different dataset. ] --- ## Ridge Regression in R **Step 0: Split into a training (including cross-validation) and held-out test set.** ```r # use caret for simple data split library(caret) set.seed(2021) trainIndex <- createDataPartition(EDsim$ED_severity, p = 0.8, list = FALSE) EDsim_train <- EDsim[trainIndex, ] EDsim_test <- EDsim[-trainIndex, ] ``` -- ```r # check data splitting dim(EDsim_train) ``` ``` ## [1] 400 19 ``` ```r dim(EDsim_test) ``` ``` ## [1] 100 19 ``` --- ## Ridge Regression in R **Step 1: Exploratory Data Analysis** <div class="container st-container"> <div style="max-height:300px;overflow-y:scroll;margin:10px 2px"> <table class="table table-striped table-bordered st-table st-table-striped st-table-bordered st-multiline "> <thead> <tr> <th align="center" class="st-protect-top-border"><strong>No</strong></th> <th align="center" class="st-protect-top-border"><strong>Variable</strong></th> <th align="center" class="st-protect-top-border"><strong>Stats / Values</strong></th> <th align="center" class="st-protect-top-border"><strong>Freqs (% of Valid)</strong></th> <th align="center" class="st-protect-top-border"><strong>Graph</strong></th> <th align="center" class="st-protect-top-border"><strong>Missing</strong></th> </tr> </thead> <tbody> <tr> <td align="center">1</td> <td align="left">ED_severity [numeric]</td> <td align="left">Mean (sd) : -4.2 (2.4) min < med < max: -9.8 < -4.1 < 2.3 IQR (CV) : 3.4 (-0.6)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABhElEQVR42u3ZTW7CMBgG4UnFHblDex56h56y3byLCEPzA8Z2mNkhoUIfxSafM/1i8NH6C/SRDIAM6TR/MVX8oMuNTeir5gcuNv9Cp91/ZXPnq9c/xTvaUb2QYU3LVHVybwBkSDIAMiQZABmSDIAMqent06WbKb8pQ6t7xjIXBSBDkgGQIckAyJBkAGRIMgAyJBkAGVKl0aqf2bEpQ0/T45pcFMCTrobRlkAlhtGWQJmLApAhdfZEu6zcd2o86u+e4TW7josCkCHJAMiQZABkSDIAu+4bxh+knsIw/iBV5qIAZEgyADIkGQAZkgyADEkGQIYkAyBD6v5kuqzGkf2ADDXmWxcFIEOSAZAhyQDIkBZ/MI94Dr2D4Yjn0GUuCkCGJAMgQ5IBkCHJAMiQZABkSDIAMiQZgKsJc9Sh+vEj+6tB+1y8YYSx+vGjABcFIEMa8KnVmrbuFgdl2LpbHJSh7Mav4Oz6eBuG/6+PaY70Peh9w74+Z1fD9Fb/+d38wQRkSDIA8Af13Sr0o5SuyAAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMNTjwTIAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTIrMDA6MDClvnmOAAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">2</td> <td align="left">emo_reg [numeric]</td> <td align="left">Mean (sd) : 4.8 (1.3) min < med < max: 1.4 < 4.8 < 8.6 IQR (CV) : 1.7 (0.3)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABXElEQVR42u3cQW4aQRBA0YfFHX2H+DzOHXLKeFOL2bix0WRgnP93CAHjJ1UD3cKXv4qXR1/AcxQDYpiu2xuXh1zC+43l6e0fXdb2Za93P8uOvS7u+3PIFTQUiGE6aChuzf+jO2xt+Hz+j5n+dQ0FYphiQAxTDIhhigExTDEghikGxDDFgBimGBDDFANimGJADFMMiGGKATFMMWDHU6tnP547iOEZjufvr6FADFMMiGGKATFMMSCGKQbEMMWAGKYYEMMUA2KYYkAM01P8umbdepdzn58gnYDhiD3OhgIxTDEghikGxDDFgG99bjj3Ye1uDOc+rF3XUCCGKQbEMMWAGKYYEMMUA2KYYsAptmTXrb7wfX3X+vQM+/zrnIYCMUwxIIYpBsQwnf4Nc9WNTeTNp4rrNx52ur6+hXy994E/q9YGxDBdtsvB7x+2Nqz7tVkiL//VX/5pDQVimGIAH7ohHLwQiFRnAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEyKzAwOjAw1OPBMgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMKW+eY4AAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">3</td> <td align="left">depression [numeric]</td> <td align="left">Mean (sd) : 2.7 (1.3) min < med < max: -1.4 < 2.7 < 5.8 IQR (CV) : 1.7 (0.5)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABXUlEQVR42u3cUW7aQBRA0UPFHrOHdj3tHrLK5ud9WJUyJMg1OLn3DyGBfaQxnhmLy1/Fj0cfwHMUA2KYrtsXl4ccwu8bl6df/+mwtl97vftTduxl8d7rIUfQoEAMUwyIYYoBMUwxIIYpBsQwxYAYpqeYWq1bz0D3mX+egOGI+WeDAjFMMSCGKQbEMMWAGKYYEMMUA2KYYkAM00ET7Vu71o/usPWG91cNjtmzXtegQAxTDIhhigExTDEghikGxDDFgBimGBDDFANimGJADFMMiGGKATFMMSCGKQbEMJ3gYeF1q93Rjz9PfXqGffZGGxSIYYoBMUwxIIZptx/MZ3+66SCGZ/hPjvtrUCCGKQbEMMWAGKYYEMN0+mWXVTfubDdrU1+a4eP3tQ0KxDB9YlCcew65G8O555DrGhSIYYoB/1wbvvJFcN1le+Z/vhXDz81d5OVbnfm7dW1ADFMM4A1aJhu8TaiamQAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMNTjwTIAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTIrMDA6MDClvnmOAAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">4</td> <td align="left">impulsivity [numeric]</td> <td align="left">Mean (sd) : 4.9 (0.8) min < med < max: 2.9 < 4.9 < 7.2 IQR (CV) : 1.1 (0.2)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABW0lEQVR42u3bUW7aQBRA0UOVPWYP7XqSPXSV6c/7IBIjEYpdu7n3D4SwObJHzNi+fCh+/OsdOEYxIIbp5frFZdNNvS2HoV/bbnjR9e68PPwtD/R6893fS6D9eHZlWLXi2a/GBsQwxYAYphgQwxQDYphiQAzTJn+m3063lrPRnOLWLGHPOcJX66RADFMMiGGKATFMMSCGKQbEMMWAGKYYcJCrVqv2u6h3aIb9Lup1UiCGKQbEMMWAGKYYEMMUA2KYYkAMUwyIYYoBMUwxIIYpBsQwxYAYpr9aoD/fjX+bMBzh8aDn1EmBGKYYEMMUA2KYYkAMUwyIYYoBMUwxIIYpBhz8vshVz79t9JQMz1/suYvhLIttt/fznmPkzqPhHIttjz8L3BCJGKYYEMMUA2KYYkAMUwyIYYoBMUyfplZnmUl+reWvupp5Xq4/8/5fMqz6uWL4vjU2IIYpBvAHt8wgun7XBHYAAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMDctMjFUMTY6MzA6MTIrMDA6MDDU48EyAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIxLTA3LTIxVDE2OjMwOjEyKzAwOjAwpb55jgAAAD10RVh0aWNjOmNvcHlyaWdodABDb3B5cmlnaHQgMjAwNyBBcHBsZSBJbmMuLCBhbGwgcmlnaHRzIHJlc2VydmVkLp5m3CkAAAAjdEVYdGljYzpkZXNjcmlwdGlvbgBHZW5lcmljIFJHQiBQcm9maWxlGqc4jgAAAABJRU5ErkJggg=="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">5</td> <td align="left">self_crit [numeric]</td> <td align="left">Mean (sd) : 4.4 (1.6) min < med < max: 0.3 < 4.3 < 8.9 IQR (CV) : 2.1 (0.3)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABX0lEQVR42u3awU3rUBBA0RNEj/QA9fB7+FXCZhYW0kOIYGKHe3cgAc5JnqUZfHlTPNz6Ao5RDIhhetx+cfm1P/v6yS3p5ZcuY3sJj9/+LVf2tPj+/5tcTYcCMUwxIIYpBsQwxYAYphgQwxQDYphiQAxTDIhhigExTDFg513k62n+F7TzSvZYi9d1HQrEMMWAGKYYEMMUA2KYYkAM082eb1i3nkT2ewDkgAy3mEM6FIhhigExTDEghikGxDDFgBimGBDDFANimGJADNMB1y7r9ttLnYphv73UDzCc5ymGXRnO8xTDum6RiGGKATFMMSCGKQbEMMWAGKYYEMP05dHqHubIH2A49hx57ULmVGuXdde+Rd0bEMMUA2KYYkAMUwyIYYoBMUwxIIYpBnyYMO9xp/DJa9qM4B8G7aflzxxhq/CdvjaCdyjAZfuZ+XeHh2Ld8+ZQXP7UK1/WoUAMUwzgHWv3H7zKKx0YAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEyKzAwOjAw1OPBMgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMKW+eY4AAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">6</td> <td align="left">anxiety [numeric]</td> <td align="left">Mean (sd) : 3.6 (0.9) min < med < max: 1.2 < 3.6 < 6.1 IQR (CV) : 1.1 (0.2)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABcklEQVR42u3bUU7bUBBA0ZOKPXYP7XroHlgl/ZmPiDpSlBgnL733D4TAOtI848GcPhU/Hn0Bz1EMiGF6O//gdOAPfr9wKP0+7CLOL+Dt5u9ydz83PvfxoGtpKBDDFANimGJADFMMiGGKATFMMeCQZ4r3BTY7hzxaPdND1HYNBWKYYkAMUwx46BJuu63b6/fvJ5+O4d+b6xG31oYCMUwxIIYpBsQwxYAYphgQwxQDYphiQAzTzg/aKyzjD2B41L7g3hoKxDDFgBimGBDDFANimGJADFMMiGGKATFMT/eH/a22txh7vvWwBMP3v1DYUCCGKQbccTasunzdmWGFN6Gvr6FADFMMiGGKATFMMSCGKQbEMMWAGKYYEMO0xC5yuz0XtQsz7LnvaCgQw3TFULzW1vFmhtfaOm7XUCCGKQZ8ORte4TC87T/+vxyR6x+Gt73B31Bg6V+mr+/CsJ+Nyun8K/68wNlwfb8uMfy/dTYghikG8BfVkiRYPWOvtwAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMNTjwTIAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTIrMDA6MDClvnmOAAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">7</td> <td align="left">race [factor]</td> <td align="left">1. 2 2. 3 3. 4 4. 5 5. 6 6. 7 7. 8</td> <td align="left" style="padding:0;vertical-align:middle"><table style="border-collapse:collapse;border:none;margin:0"><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">2</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">0.5%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">30</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">7.5%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">96</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">24.0%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">165</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">41.2%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">93</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">23.2%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">13</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">3.2%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 5px 0 7px;margin:0;border:0" align="right">1</td><td style="padding:0 2px 0 0;border:0;" align="left">(</td><td style="padding:0;border:0" align="right">0.2%</td><td style="padding:0 4px 0 2px;border:0" align="left">)</td></tr></table></td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAEEAAACUCAQAAABmDYxSAAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABDElEQVR42u3cQQrCMBQE0F/xjh7C83gIT6kbhbRoq7Wms3jjqrjIIz/QgYjDrfbOYW8AwiO35nPpdjDaVQN2AQEBAQEBAQFhKUNblIZuy7arBuxCAOHYPryqr+e/T+c4fjxNvr522IWAQSAgICAgIAQTJi/rHi/naXTHEMJid/w8a1vmQnf8POsPcsAgEBAQEBAQEOaivlblDaLftXm74Zt1x28yPnEBg0BAQEBAQECYi+5YFTGIAMJm9XX99fpG9fWXYxwwCAQEBAQEhGCC7lgVMYgAwpvu+P8fWr4hPLtj30MZMAgEBAQEBASEuaivVRGDGBE6XpsH7wICAsLeAAQEBIRIwuD/mhAeuQMJ4zAzVnxoXAAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMNTjwTIAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTIrMDA6MDClvnmOAAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">8</td> <td align="left">age [numeric]</td> <td align="left">Mean (sd) : 17.3 (1.6) min < med < max: 13.1 < 17.4 < 21.9 IQR (CV) : 2.2 (0.1)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABYklEQVR42u3ZwVHjQBBA0WeKHDcHiAdy2Ch3L30Qh6FcGGFL/H8zBxCv1PJo5vJP8XTvC3iMYkAM0/P2w+XH/uzbJ4+k1x+6jO0lPH/5t9zYn8XP/97lahoKxDDFgBimGBDDFANimGJADFMMiGGKATFMMSCGKQbEMMWAnfci3w5zFrTzluxjbbzejeErre+h/bbuH5DhHndQj0jEMMWAGKYYEMMUA2KYYkAMUwyIYYoBMUwxIIbpG7ZdjrPjuCvDcXYc1zUUiGGKATFMMSCG6QGPa9btd551KIb91icNBWKYYkAMUwyIYYoBMUwxIIYpBsQwxYAYpqtftM9wGvENDGc4jVjXUCCGKQbEMMWAGKYYcLBzinW3HuSchOHWpV1DgRimGBDD9OERee6X6asZ1k/cc7xOr2socJp1w7pPBn2zsDo9w3ULq4YCXLb3zPuv+qZ42QzF5Vf958saCsQwxQD+A6EbH8IZTcRTAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEyKzAwOjAw1OPBMgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMKW+eY4AAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">9</td> <td align="left">family_history [numeric]</td> <td align="left">Mean (sd) : 3.5 (1.8) min < med < max: -2.1 < 3.5 < 8.9 IQR (CV) : 2.4 (0.5)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABeUlEQVR42u3aQW7aUBRA0UPFHruHdj3pHrLKdPIGSLgKsrCx6b2zRIjYBz8r/5vLl+LHqw/gGMWAGKbr7Q+XlxzCx+Lt6ffmB3P7Z6+r3+WJ/bz7zefOR9BQIIYpBsQwxYAYphgQwxQDYphiQAxTDIhhigExTIfYdllqaU9qux2pwzLsuyPVUGD3q+HjoE+Hdh+K12+/LtVQIIYpBsQwxYAYphgQwxQDYphiwMZriqMupHZmuF9IHWEZtVRDgRimGBDDFANimGJADNNhH9cstd2TrFMxbLe531AghikGxDDFgBimGBDDFANimGJADFMMONkKc6n7xfeapffpGZ7zQKihwBOvhvM8r9yU4ahf/HyshgIxTDEghikGxDDFgBimGBDDFANimE6/33Dfmm9BvCHDmpVuQ4EYphiw8t5w7g23pzGce8NtqYYCD1wN7zEAi2dx87/E9dsXv8UAfHcOl9sz//MWn/yj/bq5Gi7/1Zn/s26RiGGKAfwF6E0m7vCM6M4AAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMDctMjFUMTY6MzA6MTIrMDA6MDDU48EyAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIxLTA3LTIxVDE2OjMwOjEyKzAwOjAwpb55jgAAAD10RVh0aWNjOmNvcHlyaWdodABDb3B5cmlnaHQgMjAwNyBBcHBsZSBJbmMuLCBhbGwgcmlnaHRzIHJlc2VydmVkLp5m3CkAAAAjdEVYdGljYzpkZXNjcmlwdGlvbgBHZW5lcmljIFJHQiBQcm9maWxlGqc4jgAAAABJRU5ErkJggg=="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">10</td> <td align="left">prior_treatment [numeric]</td> <td align="left">Mean (sd) : 3.1 (1.1) min < med < max: -1.1 < 3.1 < 6.2 IQR (CV) : 1.4 (0.3)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHgyea4uoAAABSklEQVR42u3cQY4SURhG0YPpPboHXU+7B1epk2+AgyJGQKr13hmEQHHIe6GoP1x+KD69+gDOUQyIYb1d37i8+mjwfmOz+vrQA7x+obc/fpan9fng/u9PfM0WBWJYMSCGFQNiWDEghhUDYlgxIIYVA152hvl+st/+Xnai/YrT6eNaFIhhxYAYVgyIYcWAGFYMiGHFgBhWDIhhxYAYVgyIYcWAGFYMiGHFgBhWDIhhxYAYVgw45ej4ccfjAPfO1n8ohucNA7QoEMOKATGsGBDDigFP/t5wtunHFzGcbfrxuBYFYlgxIIYVA2JYMSCGFQNiWDEghhUDYlgxIIYVA2JYMSCGFQNiWDHgIdcpPs5FmacynOGizL3jQB9q6Oe4ez+I9gbEsH5ZFLc3u8f+oe3f6sZ7uno/l+tHffsH9vzf78sRw/9bewNiWDGAn3BgGyKwgyYpAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEyKzAwOjAw1OPBMgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMiswMDowMKW+eY4AAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">11</td> <td align="left">perfectionism [numeric]</td> <td align="left">Mean (sd) : 3.3 (0.9) min < med < max: 0.8 < 3.4 < 6 IQR (CV) : 1.3 (0.3)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABeElEQVR42u3bW06EMABG4YOZPboHXc+4B1epL/8Dhk4kTQulnPOmkyj5Mr0RWH4weDv7AsZIBkCG9Fj/sJxyCc/i9PTZ/WLW//ZR/Vca9r75zffBV+CgAGRIMgAypIOnyOegm9bDV4rzV4VSDgpAhiQDIEOSAZAhyQDIkGQAZEgyADIkGQAZkgxA5/sNo95kOZhhe5NlhFsspRwUgAxJBkCGJAMgQ5IBkCHJAMiQZABkSDIADU+Y1zlUd2UY9cmFfTkoABmSDIAMSQZAhiQDIEMa4kWCUqVdab93LIZlOHZP6qAAZEgyADIkGQAZkgyADEkGQIYkAyBDGvZoVarfqfNSDP1OnQ4KQIYkAyBDkgGQIckAVO4brv1IRzOGaz/S0ZBhnLbfzJrt9eUZ2ry/4xQJyJBkAHbMDfMtjlUM8y2OpRwUwAQL5raaW3UTMtQM4j8M95gO/2W4x3S4g2HWit/z1XyxrD//utWg+HjFcN/cNwAyJBkA+AXa0Cb0JqWMcwAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMHKUyoYAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTMrMDA6MDADyXI6AAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">12</td> <td align="left">day_temperature [numeric]</td> <td align="left">Mean (sd) : 5 (0.4) min < med < max: 3.6 < 4.9 < 6.3 IQR (CV) : 0.6 (0.1)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABeElEQVR42u3cUW7TQBRA0RPUPbIHWE/ZA6uEn/eRGKHKJbYn4t6/SFU7PorHY0/S2y/Fl6sHsEYxIIbp7f7F7dKhvP8xTX0/dED3f+7t07/lgMP++vDq54ljuZDhysPe1tyAGKYYEMMUA2KYYkAMUwyIYYoBMUwxIIYpBsQwxYAYphhw6rPI94U3yE59JLvOI9htnRSIYYoBMUwx4OLNu4/aXmKP29pdmuG8C2wnBQ5+N6y8bjyR4fFtvdKqcVsnBWKYYkAMUwyIYYoBMUwxIIYpBsQwxYAYphgQwxQDYphiQAxTDFh8u2bbcbtYL8Vw3C5WJwVimGJADFMMiGGKATFMMeDJq8jX+XTLoQxnfyr6efcYL3VPse156M0NiGGKATFMMSCGKQbEMMWAGKZdi+mz/5Xh3h7Ht2dsO+8ptqv4te4pP/+1hX+8tVr3e7b7am5ADFMM2MwNq18J9vXh9H13bLf7n/2x1Lx/dN/+xvD/1tyAGKYYwG/VzClWeRfunAAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMHKUyoYAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTMrMDA6MDADyXI6AAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">13</td> <td align="left">rainfall [numeric]</td> <td align="left">Mean (sd) : 4.4 (0.8) min < med < max: 1.9 < 4.4 < 7.2 IQR (CV) : 1.1 (0.2)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABa0lEQVR42u3bUW7aQBRA0UOVPXYP7XqSPWSV6c/7IImjVhaGcXPvHwiBOWZsa8Zc3hQ/Hr0BaxQDYpierh9cDv+4581D0e/jP3ij60152v0uO/v56ZnXRxh8qEGBGKYYEMMUA2KYYkAMUwyIYYoBMUwxIIYpBsQwxYAYprtPwm21NUN53/nJJRgePz/ZoEAMUwyIYYoBMUwxIIZpicunre57Zbksw32vLA9leD7NjVUH/xo+7tMV7mXYqkMkYphiQAxTDIhhigExTDEghikGxDDFgBimGBDDdLNpl/PMNB3KsMK69P4aFIhhigExTDEghikGxDAtu5S71XGr3KdiOO46tUGBGKYYEMMUA2KYYkAMUwyIYYoBMUwxIIYpBuycbzj3Qt3NGM69ULdVgwIxTDEghumvh8jVzwqft2/PlP0/nCnWPivc5m9M7xhW3/N7++J7Xf1qLtevePlPGbb79RXD960zBWKYYgB/ANToI1Q3Q28CAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEzKzAwOjAwcpTKhgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMAPJcjoAAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">14</td> <td align="left">num_siblings [numeric]</td> <td align="left">Mean (sd) : 4.5 (0.6) min < med < max: 3 < 5 < 6 IQR (CV) : 1 (0.1)</td> <td align="left" style="padding:0;vertical-align:middle"><table style="border-collapse:collapse;border:none;margin:0"><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">3</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">11</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">2.8%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">4</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">180</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">45.0%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">5</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">191</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">47.8%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">6</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">18</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">4.5%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr></table></td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAEkAAABXCAQAAADipqC8AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAAAzUlEQVRo3u3awQ2EMAxEUWdFjxRBPbs9UCUcQJBkCQIO9gj9uXF7wkYaS6TJ1PKJBkB6li5/+K2LNSRvRr7R1VvqrXd/K3UEBwcJEiRIaoEECZJnuvJxjPaYWcr7nHuZ3HLSKhVSDO4bcNT99/xql7yb99HuCg4OEiRIkNQCCRIkz9C9WxHv3oKk8HNgTbYzwefAkvKjEhwcJEiQIKkFEiRInuEcaIVz4G6a54D/3zmHpP0ciFxzwcFBggQJklogQXoxSb17a0RwcJCuZAYb8Beiq3i8LQAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMHKUyoYAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTMrMDA6MDADyXI6AAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">15</td> <td align="left">num_cellphones [numeric]</td> <td align="left">Mean (sd) : 3.1 (0.9) min < med < max: 1 < 3 < 6 IQR (CV) : 1 (0.3)</td> <td align="left" style="padding:0;vertical-align:middle"><table style="border-collapse:collapse;border:none;margin:0"><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">1</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">9</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">2.2%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">2</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">87</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">21.8%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">3</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">167</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">41.8%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">4</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">121</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">30.2%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">5</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">15</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">3.8%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr><tr style="background-color:transparent"><td style="padding:0 2px 0 7px;margin:0;border:0" align="right">6</td><td style="padding:0 2px;border:0;" align="left">:</td><td style="padding:0 4px 0 6px;margin:0;border:0" align="right">1</td><td style="padding:0;border:0" align="left">(</td><td style="padding:0 2px;margin:0;border:0" align="right">0.2%</td><td style="padding:0 4px 0 0;border:0" align="left">)</td></tr></table></td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAEIAAACACAQAAAAVfXfcAAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAAA+klEQVR42u3cQQ6CMBCFYWq4I4fwPHIHT6kbq9NGiA6l85r8b8fKL50heQoxPab4XKIBIGxme7G+FuSazv9gu4vVSSzTEnASEuMAAQIECBAgQHgzl5f3EESyDadDoXpnp1nFRAJR7MTN+U3oaDGuFtNTc48vs8Q4QIAAAQIEiCERdMwciXFIIJp0TFfM7Bt0TE/KG0BiHCBAgAABAgQIbyi6OXrjaNsxf/9187SO+c92SYwDBAgQIECAAOENHTNHYhwSiM2i2+Pt1K+IT9Htu6AS4wABAgQIECCGRNAxcyTGoYfo+sBe+iRARAcECBAgQIAYEpH4rwEQVZ4CUyEzwWCWXgAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMHKUyoYAAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjEtMDctMjFUMTY6MzA6MTMrMDA6MDADyXI6AAAAPXRFWHRpY2M6Y29weXJpZ2h0AENvcHlyaWdodCAyMDA3IEFwcGxlIEluYy4sIGFsbCByaWdodHMgcmVzZXJ2ZWQunmbcKQAAACN0RVh0aWNjOmRlc2NyaXB0aW9uAEdlbmVyaWMgUkdCIFByb2ZpbGUapziOAAAAAElFTkSuQmCC"></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">16</td> <td align="left">time_news [numeric]</td> <td align="left">Mean (sd) : 4.3 (1.3) min < med < max: 1.2 < 4.2 < 8 IQR (CV) : 1.7 (0.3)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABkklEQVR42u2aS3KDMBAFm5Tv6Dsk53HukFM6m7fAohIK85NE946NwV280YzQ8ETg4+wHqAM1AGoIt/HFcMgtH5Ny9HXMjQvGj3F7+1dWcH+5+jnjEQoMBaCGoAZADUENgBqCGgA1BDUAagi7N9OPJjY0Dpgp7sV1DTNEiaEA1BDUAKghqAFQQ1ADoIagBuCknemS87fsq9BwfrttKAA1hM1D0cZgvbuGGj/NzWMogEpWiilltPZeQCvVcHSwDAWghqAGQA1BDYAaghoANQQ1AGoIagDUECodrUr2njgb0bD3xGkoADUENQBqCGoA1BDUAKghqAFQQ1jdTLf56XZzDed8ut36fEwjo1XJ1udjrA2AGsLCUPRREFdrqOHU2h4YCkANQQ2AGkKj7dOUdVv43WhY19LPaOi1T1iooc0zr8uxRAJqCGoA1BBeSuRV1oUZDb3Oj/MYCqCjLrJk2aZttxqWxbtbDVMm78fo7biQhv/GAkskoIagBkANYRiXz+9LNdOfo5ViuNQ//xNDAaghqAGAX7V+LfxhIYwuAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEzKzAwOjAwcpTKhgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMAPJcjoAAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">17</td> <td align="left">time_tv [numeric]</td> <td align="left">Mean (sd) : 4.2 (0.6) min < med < max: 2.8 < 4.2 < 5.8 IQR (CV) : 0.8 (0.1)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABUElEQVR42u3aQU4CQRBA0Yfxjt5Bz6N38JS6qcXERBsUccD/d4QEmhdqyDR9eFPc/fUC9lEMiGG63z44XOQtn4+4HD1dYCnbZdx/+1V+0MPi+deLr6ihQAxTDIhhigExTDEghikGxDDFgBimGBDDFANimGJADFMM+IW9yGM2XPfXL2zJ7m/DdV1DgT/aoF+3Gq1z/4+xU4avB+v8Y9VQIIYpBsQwxYAYphgQwxQDYphiQAxTDIhhigExTDEghikGxDDFgBimGBDDFANimGJADFMMiGGKATFMMSCGKQbEMMWA3Z6EW7U+i3vakcErZTj3SdwTGa7zRPTZGa7zRPS6LpGIYYoBMUwxIIYpBsQwxYAYphgQwxQDPtxh3upt9IkMt3obva6hwNVuwq07YsA3u5U3y3DaeDcU4LD97rz8q1+Kx81QHP7VJ/+0hgIxTDGAd1F5GibITqWvAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIxLTA3LTIxVDE2OjMwOjEzKzAwOjAwcpTKhgAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMS0wNy0yMVQxNjozMDoxMyswMDowMAPJcjoAAAA9dEVYdGljYzpjb3B5cmlnaHQAQ29weXJpZ2h0IDIwMDcgQXBwbGUgSW5jLiwgYWxsIHJpZ2h0cyByZXNlcnZlZC6eZtwpAAAAI3RFWHRpY2M6ZGVzY3JpcHRpb24AR2VuZXJpYyBSR0IgUHJvZmlsZRqnOI4AAAAASUVORK5CYII="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">18</td> <td align="left">num_pets [numeric]</td> <td align="left">Mean (sd) : 2.1 (2.5) min < med < max: -5 < 2 < 9 IQR (CV) : 4 (1.2)</td> <td align="left" style="vertical-align:middle">15 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABkUlEQVR42u3by1HDMAAG4TWTHtMD1BN6oEq4/AdHGvAYvyRl9+YLcT4sWZGT6RuDt6tPoI1kAGRIt/nBdMpLPqrp6OOcFy6an8bt339lQ/eno68rTqHIQQGccDU8urgjnzAo7sVxC4OgzEEByJAuuVOUXX8LbYLh+tnDQQHIkGQAZEgyADIkGQAZkgxAI6vIunJ5ffTieneGffYXzt6fOuBqaG+LbTnnBkCGJAMgQ5IBkCHJAMiQZABkSDIAMiQZABmSDIAMSQZAhiQDIEOSAZAhNfq4puzoxzedMBz97MNBAexwNfTxZeDDGfp8WFfmoABkSDIAMiQZABmSDIAMSQZAhiQDIEOSAZAhyQDIkFZuu4yx17SZoYWfjsL+v+PtZGe6rP5nbNvC75ShbtuOqFMkIEOSAZAhyQDIkGQAFtcNoy6eVzKM8aB2OQcFIEN6GhQjzQTrPoMWc0MbH6P3aN07cVAAMiQZgIG2XZarJs3ZlPlCDH8tBKc50edAN8zl3mdXw/RS7/zXnCIBGZIMAPwAhXMt9AnFnp4AAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMDctMjFUMTY6MzA6MTMrMDA6MDBylMqGAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIxLTA3LTIxVDE2OjMwOjEzKzAwOjAwA8lyOgAAAD10RVh0aWNjOmNvcHlyaWdodABDb3B5cmlnaHQgMjAwNyBBcHBsZSBJbmMuLCBhbGwgcmlnaHRzIHJlc2VydmVkLp5m3CkAAAAjdEVYdGljYzpkZXNjcmlwdGlvbgBHZW5lcmljIFJHQiBQcm9maWxlGqc4jgAAAABJRU5ErkJggg=="></td> <td align="center">0 (0.0%)</td> </tr> <tr> <td align="center">19</td> <td align="left">treatment_length [numeric]</td> <td align="left">Mean (sd) : 5.8 (1.1) min < med < max: 2.7 < 5.8 < 9.4 IQR (CV) : 1.4 (0.2)</td> <td align="left" style="vertical-align:middle">400 distinct values</td> <td align="left" style="vertical-align:middle;padding:0;background-color:transparent"><img style="border:none;background-color:transparent;padding:0" src="data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAAAIMAAABfCAQAAABhaLy5AAAAAmJLR0QA/4ePzL8AAAAHdElNRQflBxUQHg3pbLs+AAABi0lEQVR42u3bQXaCMAAG4aHPO3qH9jz2Dp6y3fwLDFYehUCCMzs2Cp8EAsLwg8HH0SvQRjIAMqTLeGHY5Stvk8PR1z5fXDRejcu/P2VF14el+xGrUOSgAGRIMgAyJBkAGZIMgAxJBmCHWeStiyv5HSbT12K5hcnzAQzzHX+x1QTD8fuLh0hAhiQDIEOSAZAhyQDIkGQAZEgyADIkGQAZkgyADEkGQIYkAyBDauKW7LTyXnXtO9WNMuz9WJCDApAhyQDIkGQAZEibnzD7eJ6hOkOLjwLP56AAZEiNTqbLal9jdMJQ+3jjoABkSDIAMiQZgA3OFH1Onjdn6HPyXOagAGRIMgAyJBkAGZIMgAxJBkCGJAMgQ5IBkCHJAHRzg75s6/d4O2XY+j1eBwUgQ1o4KM5xA3Y1Qwtv19fIQQHIkGQAup03TFv3IMhpGOb+O3tyjhtBzTCc6QT5CuqB4dlG9/oP5bIfsNgbzjMrWLYlnikAGMb7zveJjgTzfY4OkcNbbfmfOSgAGZIMAPwCAoYtiMfbc1sAAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMDctMjFUMTY6MzA6MTMrMDA6MDBylMqGAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIxLTA3LTIxVDE2OjMwOjEzKzAwOjAwA8lyOgAAAD10RVh0aWNjOmNvcHlyaWdodABDb3B5cmlnaHQgMjAwNyBBcHBsZSBJbmMuLCBhbGwgcmlnaHRzIHJlc2VydmVkLp5m3CkAAAAjdEVYdGljYzpkZXNjcmlwdGlvbgBHZW5lcmljIFJHQiBQcm9maWxlGqc4jgAAAABJRU5ErkJggg=="></td> <td align="center">0 (0.0%)</td> </tr> </tbody> </table> </div> <p>Generated by <a href='https://github.com/dcomtois/summarytools'>summarytools</a> 0.9.9 (<a href='https://www.r-project.org/'>R</a> version 4.1.0)<br/>2021-07-21</p> </div> --- ## Ridge Regression in R **Step 1: Exploratory Data Analysis** <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-12-1.png" width="100%" /> --- ## Ridge Regression in R **Step 2: Feature Engineering**¹. Normalizing (standardizing) features before regularization is important to ensure that the shrinkage parameter `\(\lambda\)` affects all features equally. ```r EDsim_recipe <- EDsim_train %>% recipe(ED_severity ~ .) %>% * step_normalize(all_numeric_predictors()) %>% step_dummy(all_nominal_predictors()) ``` .footnote[ [1] Note than when using `recipes`, dummy variables should be created manually with `step_dummy`. If using `caret` by itself without `recipes`, dummy variables are created automatically. ] --- ## Ridge Regression in R **Step 3: Set model training and tuning methods in `caret::traincontrol()`** ```r EDsim_control <- trainControl(method = 'repeatedcv', number = 10, repeats = 3) ``` <br/> This sets our up training procedure with the following specifications: - 10-fold cross-validation, repeated 3 times - Other commonly used options include `method = 'cv'` for a single (non-repeated) cross-validation and `method = 'boot'` for bootstrapping --- ## Ridge Regression in R **Step 4: Set a tuning grid** (optional) There are several ways to set a tuning grid in `caret`. The first option is to do nothing, as `caret` will automatically create a tuning grid if one is not specified. By default, `caret::train()` creates a tuning grid of size `\(3^P\)`, where `\(P\)` is the number of tuning parameters. -- </br> The second option is to specify the length of the tuning grid in the `caret::train()` function: ```r train(Y ~ X, method = 'glmnet', tuneLength = 10) ``` --- ## Ridge Regression in R **Step 4: Set a tuning grid** (optional) The third option is to specify the grid yourself to call in `caret::train()`: ```r ridgegrid <- c(0, 0.2, 0.4, 0.6, 0.8, 1.0) ridgegrid ``` ``` ## [1] 0.0 0.2 0.4 0.6 0.8 1.0 ``` -- The `seq()` function is a useful shortcut for creating a sequence of numbers: ```r ridgegrid <- seq(0, 1, 0.2) ridgegrid ``` ``` ## [1] 0.0 0.2 0.4 0.6 0.8 1.0 ``` --- ## Ridge Regression in R **Step 4: Set a tuning grid** (optional) When using `method = glmnet` in {caret}, we need to create a grid of `\(\lambda\)` *and* `\(\alpha\)` values, holding `\(\alpha\)` constant at 0 for ridge regression. The `expand.grid()` function is useful for doing so. .scroll-output[ ```r ridgegrid <- expand.grid(alpha = 0, lambda = seq(0, 1, 0.01)) ridgegrid ``` ``` ## alpha lambda ## 1 0 0.00 ## 2 0 0.01 ## 3 0 0.02 ## 4 0 0.03 ## 5 0 0.04 ## 6 0 0.05 ## 7 0 0.06 ## 8 0 0.07 ## 9 0 0.08 ## 10 0 0.09 ## 11 0 0.10 ## 12 0 0.11 ## 13 0 0.12 ## 14 0 0.13 ## 15 0 0.14 ## 16 0 0.15 ## 17 0 0.16 ## 18 0 0.17 ## 19 0 0.18 ## 20 0 0.19 ## 21 0 0.20 ## 22 0 0.21 ## 23 0 0.22 ## 24 0 0.23 ## 25 0 0.24 ## 26 0 0.25 ## 27 0 0.26 ## 28 0 0.27 ## 29 0 0.28 ## 30 0 0.29 ## 31 0 0.30 ## 32 0 0.31 ## 33 0 0.32 ## 34 0 0.33 ## 35 0 0.34 ## 36 0 0.35 ## 37 0 0.36 ## 38 0 0.37 ## 39 0 0.38 ## 40 0 0.39 ## 41 0 0.40 ## 42 0 0.41 ## 43 0 0.42 ## 44 0 0.43 ## 45 0 0.44 ## 46 0 0.45 ## 47 0 0.46 ## 48 0 0.47 ## 49 0 0.48 ## 50 0 0.49 ## 51 0 0.50 ## 52 0 0.51 ## 53 0 0.52 ## 54 0 0.53 ## 55 0 0.54 ## 56 0 0.55 ## 57 0 0.56 ## 58 0 0.57 ## 59 0 0.58 ## 60 0 0.59 ## 61 0 0.60 ## 62 0 0.61 ## 63 0 0.62 ## 64 0 0.63 ## 65 0 0.64 ## 66 0 0.65 ## 67 0 0.66 ## 68 0 0.67 ## 69 0 0.68 ## 70 0 0.69 ## 71 0 0.70 ## 72 0 0.71 ## 73 0 0.72 ## 74 0 0.73 ## 75 0 0.74 ## 76 0 0.75 ## 77 0 0.76 ## 78 0 0.77 ## 79 0 0.78 ## 80 0 0.79 ## 81 0 0.80 ## 82 0 0.81 ## 83 0 0.82 ## 84 0 0.83 ## 85 0 0.84 ## 86 0 0.85 ## 87 0 0.86 ## 88 0 0.87 ## 89 0 0.88 ## 90 0 0.89 ## 91 0 0.90 ## 92 0 0.91 ## 93 0 0.92 ## 94 0 0.93 ## 95 0 0.94 ## 96 0 0.95 ## 97 0 0.96 ## 98 0 0.97 ## 99 0 0.98 ## 100 0 0.99 ## 101 0 1.00 ``` ] --- ## Ridge Regression in R **Step 5: Train your model using `caret::train()`** ```r set.seed(2021) ridgefit <- train(EDsim_recipe, data = EDsim_train, method = 'glmnet', tuneGrid = ridgegrid, trControl = EDsim_control) ``` -- .scroll-output[ ```r ridgefit ``` ``` ## glmnet ## ## 400 samples ## 18 predictor ## ## Recipe steps: normalize, dummy ## Resampling: Cross-Validated (10 fold, repeated 3 times) ## Summary of sample sizes: 360, 360, 360, 360, 360, 360, ... ## Resampling results across tuning parameters: ## ## lambda RMSE Rsquared MAE ## 0.00 0.9858604 0.8273531 0.8502405 ## 0.01 0.9858604 0.8273531 0.8502405 ## 0.02 0.9858604 0.8273531 0.8502405 ## 0.03 0.9858604 0.8273531 0.8502405 ## 0.04 0.9858604 0.8273531 0.8502405 ## 0.05 0.9858604 0.8273531 0.8502405 ## 0.06 0.9858604 0.8273531 0.8502405 ## 0.07 0.9858604 0.8273531 0.8502405 ## 0.08 0.9858604 0.8273531 0.8502405 ## 0.09 0.9858604 0.8273531 0.8502405 ## 0.10 0.9858604 0.8273531 0.8502405 ## 0.11 0.9858604 0.8273531 0.8502405 ## 0.12 0.9858533 0.8273545 0.8502622 ## 0.13 0.9860350 0.8273537 0.8507529 ## 0.14 0.9863098 0.8273474 0.8513872 ## 0.15 0.9866312 0.8273356 0.8520245 ## 0.16 0.9869937 0.8273194 0.8526694 ## 0.17 0.9873916 0.8273003 0.8533233 ## 0.18 0.9878303 0.8272764 0.8539814 ## 0.19 0.9883039 0.8272495 0.8546488 ## 0.20 0.9888131 0.8272188 0.8553413 ## 0.21 0.9893571 0.8271850 0.8560311 ## 0.22 0.9899335 0.8271478 0.8567300 ## 0.23 0.9905429 0.8271075 0.8574358 ## 0.24 0.9911822 0.8270643 0.8581651 ## 0.25 0.9918510 0.8270186 0.8589209 ## 0.26 0.9925508 0.8269692 0.8597141 ## 0.27 0.9932754 0.8269180 0.8605379 ## 0.28 0.9940331 0.8268624 0.8613605 ## 0.29 0.9948150 0.8268046 0.8621802 ## 0.30 0.9956232 0.8267446 0.8630183 ## 0.31 0.9964582 0.8266816 0.8638602 ## 0.32 0.9973134 0.8266171 0.8647131 ## 0.33 0.9981945 0.8265500 0.8656071 ## 0.34 0.9990999 0.8264799 0.8665168 ## 0.35 1.0000238 0.8264084 0.8674594 ## 0.36 1.0009711 0.8263345 0.8684351 ## 0.37 1.0019424 0.8262576 0.8694199 ## 0.38 1.0029307 0.8261792 0.8704097 ## 0.39 1.0039375 0.8260994 0.8714098 ## 0.40 1.0049675 0.8260167 0.8724188 ## 0.41 1.0060162 0.8259315 0.8734519 ## 0.42 1.0070795 0.8258450 0.8744980 ## 0.43 1.0081612 0.8257568 0.8755388 ## 0.44 1.0092638 0.8256657 0.8765937 ## 0.45 1.0103821 0.8255725 0.8776601 ## 0.46 1.0115120 0.8254785 0.8787585 ## 0.47 1.0126583 0.8253829 0.8798701 ## 0.48 1.0138232 0.8252852 0.8809847 ## 0.49 1.0150026 0.8251853 0.8821182 ## 0.50 1.0161933 0.8250844 0.8832541 ## 0.51 1.0173952 0.8249828 0.8843797 ## 0.52 1.0186134 0.8248792 0.8855160 ## 0.53 1.0198476 0.8247733 0.8866858 ## 0.54 1.0210933 0.8246656 0.8878775 ## 0.55 1.0223476 0.8245569 0.8890777 ## 0.56 1.0236123 0.8244474 0.8902915 ## 0.57 1.0248914 0.8243363 0.8915214 ## 0.58 1.0261847 0.8242230 0.8927962 ## 0.59 1.0274867 0.8241084 0.8940963 ## 0.60 1.0287969 0.8239926 0.8954005 ## 0.61 1.0301141 0.8238765 0.8966966 ## 0.62 1.0314425 0.8237590 0.8980012 ## 0.63 1.0327839 0.8236396 0.8993304 ## 0.64 1.0341356 0.8235182 0.9006650 ## 0.65 1.0354947 0.8233956 0.9020130 ## 0.66 1.0368591 0.8232722 0.9033624 ## 0.67 1.0382290 0.8231486 0.9047228 ## 0.68 1.0396090 0.8230237 0.9060934 ## 0.69 1.0409999 0.8228970 0.9074756 ## 0.70 1.0424005 0.8227684 0.9088791 ## 0.71 1.0438055 0.8226388 0.9102821 ## 0.72 1.0452152 0.8225083 0.9116753 ## 0.73 1.0466277 0.8223778 0.9130562 ## 0.74 1.0480472 0.8222465 0.9144352 ## 0.75 1.0494759 0.8221139 0.9158265 ## 0.76 1.0509128 0.8219797 0.9172239 ## 0.77 1.0523554 0.8218441 0.9186234 ## 0.78 1.0538016 0.8217076 0.9200161 ## 0.79 1.0552508 0.8215704 0.9214157 ## 0.80 1.0567016 0.8214332 0.9228224 ## 0.81 1.0581576 0.8212954 0.9242229 ## 0.82 1.0596206 0.8211565 0.9256200 ## 0.83 1.0610905 0.8210162 0.9270142 ## 0.84 1.0625683 0.8208741 0.9284073 ## 0.85 1.0640450 0.8207315 0.9297997 ## 0.86 1.0655263 0.8205880 0.9312021 ## 0.87 1.0670075 0.8204441 0.9326083 ## 0.88 1.0684893 0.8203004 0.9340102 ## 0.89 1.0699761 0.8201561 0.9354220 ## 0.90 1.0714684 0.8200107 0.9368459 ## 0.91 1.0729661 0.8198640 0.9382642 ## 0.92 1.0744710 0.8197158 0.9396790 ## 0.93 1.0759726 0.8195672 0.9410794 ## 0.94 1.0774771 0.8194177 0.9424723 ## 0.95 1.0789820 0.8192678 0.9438559 ## 0.96 1.0804867 0.8191179 0.9452291 ## 0.97 1.0819919 0.8189680 0.9465922 ## 0.98 1.0835006 0.8188175 0.9479490 ## 0.99 1.0850148 0.8186659 0.9493008 ## 1.00 1.0865316 0.8185134 0.9506467 ## ## Tuning parameter 'alpha' was held constant at a value of 0 ## RMSE was used to select the optimal model using ## the smallest value. ## The final values used for the model were alpha = 0 ## and lambda = 0.12. ``` ] --- ## Ridge Regression in R **Step 6: Predict on held-out test set** ```r EDsim_test_baked <- EDsim_recipe %>% * prep(training = EDsim_train) %>% * bake(new_data = EDsim_test) ridgepred_test <- predict(ridgefit, newdata = EDsim_test) postResample(pred = ridgepred_test, obs = EDsim_test_baked$ED_severity) ``` ``` ## RMSE Rsquared MAE ## 0.9425573 0.8630514 0.8213249 ``` --- ## Ridge Regression in R Accuracy as a function of `\(\lambda\)`: as `\(\lambda\)` increases to a certain point there's a small (nearly undetectable) drop in RMSE, but then RMSE starts increasing past a certain point. ```r ggplot(ridgefit) ``` <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-23-1.png" width="95%" /> --- ## Ridge Regression in R Coefficients as a function of `\(\lambda\)`: all coefficients shrink towards 0 as `\(\lambda\)` increases, but never actually reach 0. ```r library(coefplot) coefpath(ridgefit$finalModel) ``` <div id="coefpath_iC4oBJULWg8m" style="width:100%;height:252px;" class="dygraphs html-widget"></div> <script type="application/json" data-for="coefpath_iC4oBJULWg8m">{"x":{"attrs":{"axes":{"x":{"pixelsPerLabel":60},"y":[]},"labels":["lambda","emo_reg","depression","impulsivity","self_crit","anxiety","age","family_history","prior_treatment","perfectionism","day_temperature","rainfall","num_siblings","num_cellphones","time_news","time_tv","num_pets","treatment_length","race_X3","race_X4","race_X5","race_X6","race_X7","race_X8"],"retainDateWindow":false,"xlabel":"Log Lambda","ylabel":"Coefficients","legend":"onmouseover","labelsDivWidth":250,"labelsShowZeroValues":true,"labelsSeparateLines":false,"showRangeSelector":true,"rangeSelectorHeight":40,"rangeSelectorPlotFillColor":" #A7B1C4","rangeSelectorPlotStrokeColor":"#808FAB","interactionModel":"Dygraph.Interaction.defaultModel","highlightCircleSize":3,"highlightSeriesBackgroundAlpha":0.5,"highlightSeriesOpts":{"strokeWidth":3},"hideOverlayOnMouseOut":true},"annotations":[{"x":-2.1082594326026,"shortText":"emo_reg","text":"emo_reg","width":50,"attachAtBottom":false,"series":"emo_reg"},{"x":-2.1082594326026,"shortText":"depression","text":"depression","width":50,"attachAtBottom":false,"series":"depression"},{"x":-2.1082594326026,"shortText":"impulsivity","text":"impulsivity","width":50,"attachAtBottom":false,"series":"impulsivity"},{"x":-2.1082594326026,"shortText":"self_crit","text":"self_crit","width":50,"attachAtBottom":false,"series":"self_crit"},{"x":-2.1082594326026,"shortText":"anxiety","text":"anxiety","width":50,"attachAtBottom":false,"series":"anxiety"},{"x":-2.1082594326026,"shortText":"age","text":"age","width":50,"attachAtBottom":false,"series":"age"},{"x":-2.1082594326026,"shortText":"family_history","text":"family_history","width":50,"attachAtBottom":false,"series":"family_history"},{"x":-2.1082594326026,"shortText":"prior_treatment","text":"prior_treatment","width":50,"attachAtBottom":false,"series":"prior_treatment"},{"x":-2.1082594326026,"shortText":"perfectionism","text":"perfectionism","width":50,"attachAtBottom":false,"series":"perfectionism"},{"x":-2.1082594326026,"shortText":"day_temperature","text":"day_temperature","width":50,"attachAtBottom":false,"series":"day_temperature"},{"x":-2.1082594326026,"shortText":"rainfall","text":"rainfall","width":50,"attachAtBottom":false,"series":"rainfall"},{"x":-2.1082594326026,"shortText":"num_siblings","text":"num_siblings","width":50,"attachAtBottom":false,"series":"num_siblings"},{"x":-2.1082594326026,"shortText":"num_cellphones","text":"num_cellphones","width":50,"attachAtBottom":false,"series":"num_cellphones"},{"x":-2.1082594326026,"shortText":"time_news","text":"time_news","width":50,"attachAtBottom":false,"series":"time_news"},{"x":-2.1082594326026,"shortText":"time_tv","text":"time_tv","width":50,"attachAtBottom":false,"series":"time_tv"},{"x":-2.1082594326026,"shortText":"num_pets","text":"num_pets","width":50,"attachAtBottom":false,"series":"num_pets"},{"x":-2.1082594326026,"shortText":"treatment_length","text":"treatment_length","width":50,"attachAtBottom":false,"series":"treatment_length"},{"x":-2.1082594326026,"shortText":"race_X3","text":"race_X3","width":50,"attachAtBottom":false,"series":"race_X3"},{"x":-2.1082594326026,"shortText":"race_X4","text":"race_X4","width":50,"attachAtBottom":false,"series":"race_X4"},{"x":-2.1082594326026,"shortText":"race_X5","text":"race_X5","width":50,"attachAtBottom":false,"series":"race_X5"},{"x":-2.1082594326026,"shortText":"race_X6","text":"race_X6","width":50,"attachAtBottom":false,"series":"race_X6"},{"x":-2.1082594326026,"shortText":"race_X7","text":"race_X7","width":50,"attachAtBottom":false,"series":"race_X7"},{"x":-2.1082594326026,"shortText":"race_X8","text":"race_X8","width":50,"attachAtBottom":false,"series":"race_X8"}],"shadings":[],"events":[],"format":"numeric","data":[[-2.1082594326026,-2.01522569155817,-1.92219195051374,-1.82915820946931,-1.73612446842488,-1.64309072738046,-1.55005698633603,-1.4570232452916,-1.36398950424717,-1.27095576320274,-1.17792202215832,-1.08488828111389,-0.99185454006946,-0.898820799025032,-0.805787057980604,-0.712753316936176,-0.619719575891748,-0.526685834847319,-0.433652093802891,-0.340618352758463,-0.247584611714035,-0.154550870669607,-0.0615171296251788,0.0315166114192494,0.124550352463678,0.217584093508105,0.310617834552534,0.403651575596962,0.49668531664139,0.589719057685818,0.682752798730246,0.775786539774674,0.868820280819102,0.96185402186353,1.05488776290796,1.14792150395239,1.24095524499681,1.33398898604124,1.42702272708567,1.5200564681301,1.61309020917453,1.70612395021895,1.79915769126338,1.89219143230781,1.98522517335224,2.07825891439667,2.1712926554411,2.26432639648552,2.35736013752995,2.45039387857438,2.54342761961881,2.63646136066324,2.72949510170766,2.82252884275209,2.91556258379652,3.00859632484095,3.10163006588538,3.19466380692981,3.28769754797423,3.38073128901866,3.47376503006309,3.56679877110752,3.65983251215195,3.75286625319637,3.8458999942408,3.93893373528523,4.03196747632966,4.12500121737409,4.21803495841851,4.31106869946294,4.40410244050737,4.4971361815518,4.59016992259623,4.68320366364066,4.77623740468508,4.86927114572951,4.96230488677394,5.05533862781837,5.1483723688628,5.24140610990722,5.33443985095165,5.42747359199608,5.52050733304051,5.61354107408494,5.70657481512936,5.79960855617379,5.89264229721822,5.98567603826265,6.07870977930708,6.17174352035151,6.26477726139593,6.35781100244036,6.45084474348479,6.54387848452922,6.63691222557365,6.72994596661807,6.8229797076625,6.91601344870693,7.00904718975136,7.10208093079579],[0.281758098010974,0.278188657383197,0.274773966362301,0.271463071532151,0.268236376603608,0.265598569631237,0.26315395591418,0.260877885247764,0.258711737908603,0.256649682515953,0.254974951428208,0.253438389215788,0.252016906290268,0.250663305193301,0.249446975420824,0.248307043529751,0.247198200546228,0.246099801818008,0.244991093588466,0.243729911123729,0.242519569068086,0.241233989026179,0.239851220574577,0.238350143500866,0.236710641200138,0.234913734233444,0.232941669180871,0.230777965664099,0.228407431840857,0.225816211652628,0.222992968257199,0.219871953722922,0.216532982029785,0.212938796519145,0.209085813506718,0.204972747967544,0.200600897075177,0.195974432436415,0.191100648373312,0.185990118741381,0.180656732046377,0.175117596187498,0.169392822573294,0.163505210604845,0.157479857349255,0.15134371576868,0.145125120916842,0.138853299337903,0.132557873799029,0.126268373834683,0.120013762117643,0.113821986825576,0.107719570324277,0.101731244154941,0.0958796392290298,0.0901850382664659,0.0846651950114243,0.0793352218892993,0.0742075448317026,0.0692919212755439,0.0645955150575103,0.0601230202165827,0.0558768246436649,0.0518577604703074,0.0480629872699085,0.0444898885976297,0.0411337351554706,0.0379885848620851,0.0350475039089198,0.032302774818603,0.0297460884565923,0.0273687180135169,0.0251616739061536,0.0231158393244967,0.0212220867781709,0.0194713764740728,0.0178548377016497,0.0163638346295301,0.0149900180459618,0.0137253646244307,0.0125622052826711,0.0114932441439688,0.0105115695180247,0.0096106582061985,0.00878437431199286,0.00802682430467463,0.00733293813139558,0.00669751462603094,0.00611588133459752,0.00558369683820489,0.00509693457264096,0.00465186608195316,0.0042450440408838,0.00387328531545315,0.00353365427458643,0.00322344651764658,0.00294017314221354,0.00268154564259772,0.00244546551159019,1.17115229180404e-36],[0.298825869757788,0.296183236044893,0.293362225876898,0.290356392982517,0.287197754786717,0.284085148711522,0.280818399794906,0.277383823381878,0.273766272539547,0.2699605820361,0.26625023029432,0.262430781825487,0.258502304346654,0.254453830276376,0.250232617248462,0.246160277273976,0.242044145175179,0.237898589097589,0.233735981700493,0.229587535473819,0.2253846838294,0.22119702621813,0.217032317150211,0.21289664794423,0.208794334724769,0.204727840927466,0.200697724476163,0.196702597810598,0.192739086024197,0.188801941886541,0.184880907792856,0.181052143259964,0.177121745085212,0.173176928944853,0.169206457401218,0.16519870750707,0.161142137451961,0.157025840014056,0.152840142159156,0.148577194208853,0.144231490732359,0.139800276980537,0.135283813267672,0.130685488698777,0.126011790676027,0.121272146176848,0.116478655472762,0.111645740303505,0.106789728118097,0.10192839297412,0.0970804726150789,0.0922651802309305,0.0875017282875988,0.0828088803460654,0.078204544796946,0.0737054218600058,0.0693267121295215,0.0650818915853566,0.060982554592052,0.0570383232316632,0.053256818576352,0.0496436873523247,0.0462026759503012,0.0429356915756584,0.0398431909992357,0.0369238903034232,0.0341753169847417,0.0315938667113565,0.0291749829789017,0.0269133306445737,0.0248029595956768,0.0228374558815477,0.021010078591276,0.0193138815741904,0.0177418197690975,0.0162868404351807,0.0149419599727672,0.013700327300742,0.0125552749363199,0.0115003590197994,0.0105293895587421,0.00963645214819972,0.00881592236968475,0.00806247399298105,0.00737108201097396,0.00673694938902805,0.00615580758418494,0.00562342215618753,0.00513593122771605,0.00468974051735018,0.00428151138994118,0.00390814819165517,0.00356678518221893,0.00325477331868084,0.0029696670945456,0.00270921159486168,0.00247132989107497,0.00225411086852642,0.00205579965921035,9.85228313666915e-37],[-0.00186288210841505,0.00607406419898873,0.0139221389359448,0.0217435377963836,0.0297605828679787,0.0370978413546814,0.0442573794354171,0.0512359042005546,0.0580770117673208,0.0650104607453568,0.0713378717857681,0.0774442067783308,0.0833192841885664,0.088977626810965,0.0946050083462922,0.0997274713223589,0.104600559890294,0.109215993951384,0.113562855192951,0.117603423298531,0.121409297850689,0.124921117825458,0.128132147796307,0.131036051024783,0.133627001583439,0.135899811856095,0.137850065617102,0.139474238594468,0.140769781238882,0.141735222600432,0.142366824184429,0.14277021197949,0.142741977200499,0.142380823160266,0.141689663798273,0.140672578149995,0.139334809248032,0.13768283316901,0.135724509939585,0.133469301120354,0.130928517024743,0.1281155465493,0.125046025395608,0.121737910331411,0.118211442707208,0.114488999217111,0.110594839331727,0.106554766248937,0.102395722094158,0.0981453394108258,0.0938314706469413,0.0894817160403668,0.0851229684159973,0.0807809910934788,0.0764800424166748,0.0722425573946119,0.0680888936720005,0.0640371456693926,0.0601030274352585,0.0562998217313714,0.05263839030146,0.0491272382744101,0.04577262428896,0.0425785440896483,0.0395476127072579,0.0366800930198283,0.0339749589229362,0.0314298572706365,0.0290413103672909,0.0268049103590879,0.0247155018122033,0.0227673498699106,0.0209542923599901,0.0192698750569768,0.0177074699851044,0.0162603771815217,0.0149219107368808,0.0136854702075238,0.0125445986681964,0.0114930287648663,0.0105247181514116,0.00963387566767673,0.00881497955363357,0.00806278890691478,0.00737234948841614,0.0067389656439869,0.00615832133309468,0.00562627736392533,0.00513900399039546,0.00469293327472244,0.00428474811078421,0.00391137037677356,0.00356994855317425,0.00325784508057454,0.00297262367849086,0.00271203680057455,0.00247401336262041,0.00225664684696626,0.00205818474795655,9.86732969062746e-37],[0.341302859084064,0.338665975350993,0.336091727973968,0.333568475973746,0.331002137234761,0.328506843710645,0.326077232168197,0.323721239200911,0.321438791625547,0.319174122322907,0.316903846956913,0.314664694298619,0.312452773476077,0.310269721122333,0.308121600972728,0.305855167720801,0.303546555921878,0.301177244634774,0.298724935241126,0.296131118083532,0.29348849265201,0.290711180424887,0.287780273991913,0.284677493061012,0.281385516680233,0.277888330003688,0.274171584572351,0.270222966217795,0.266032558699796,0.261593123904781,0.256900849603345,0.251970047781285,0.246778696373289,0.241338384013572,0.235656001952033,0.229741359871765,0.223606969989125,0.21726777676776,0.21074087346372,0.204045239645355,0.19720151698548,0.19023182162183,0.183159577087729,0.176009345584841,0.168806636851036,0.161577680468243,0.154349156051364,0.147147883905848,0.140000485042698,0.132933023442982,0.125970645328298,0.119137230355094,0.112455068587479,0.10594457521697,0.0996240525866317,0.0935095063586849,0.0876145198122429,0.0819501874285493,0.0765251062655045,0.071345421280404,0.0664149188439877,0.0617351612770127,0.0573056543645482,0.0531239716948433,0.0491862533576479,0.0454869536171409,0.0420193989629008,0.0387759092461112,0.0357480023959146,0.032926586152019,0.0303021341044117,0.0278648443190151,0.0256047796884825,0.0235119898655676,0.0215766152101693,0.0197889736167802,0.0181396313992772,0.016619459610971,0.0152196772872905,0.0139318831341817,0.0127480771639032,0.0116606737164,0.0106625072117036,0.00974683186760865,0.00890731649580577,0.00813802977235365,0.00743345159434439,0.00678842120575217,0.00619814745044098,0.00565818437770655,0.00516441322673713,0.00471302409643193,0.00430049760040468,0.00392358674615097,0.00357929922511739,0.00326488025601551,0.00297779608637235,0.00271571822619914,0.00247650866306257,1.18544702317786e-36],[0.400718395004935,0.39942547234146,0.398008835109008,0.396461579465919,0.394644549364532,0.392561367709523,0.390313277907806,0.387913715744065,0.385372947822323,0.382545456236059,0.379489297180133,0.376246661690706,0.372827134358647,0.369245458212154,0.365335732998446,0.361302288316442,0.357084009618179,0.352682148771382,0.348102833171646,0.343460463316399,0.338542116799986,0.333453566934895,0.328198801748764,0.322781524116082,0.317205006598292,0.311471996301675,0.305584691374088,0.29954480729594,0.29335374598999,0.287012744655158,0.280525421949,0.273827723419882,0.267067057508987,0.260166181254678,0.253128987600776,0.24596086860352,0.238668894903646,0.231261907619608,0.223750542563947,0.21614721357966,0.208466076336392,0.200722981012803,0.192935408854463,0.185122378932408,0.177304309521385,0.169502822442275,0.161740486039009,0.154040500610115,0.146426337171293,0.138921345363401,0.131548348879575,0.124329247199598,0.117284641153989,0.110433497402635,0.10379286375546,0.0973776437502498,0.0912004353192949,0.0852714349440437,0.0795984055909134,0.0741867040764917,0.0690393614202843,0.0641572082572415,0.0595390365048123,0.055181759517404,0.0510807393718045,0.0472298145824778,0.0436216514368205,0.0402479255636576,0.0370995283167304,0.034166760238452,0.0314395087443803,0.0289074082041196,0.0265599814936112,0.0243867628428265,0.0223774024000449,0.0205217533866769,0.0188099430386395,0.0172324287388071,0.0157800408587041,0.0144440138649344,0.013216007224231,0.0120881175759962,0.0110528835461358,0.0101032844619592,0.00923273410380434,0.00843506937143035,0.00770454170301515,0.00703579417882861,0.00642384933777181,0.00586408952332615,0.00535223788185409,0.00488433908587766,0.00445674008569498,0.00406607113058122,0.00370922724753486,0.00338335032028338,0.00308581187322791,0.00281419663335828,0.00256628697736639,1.22829564348962e-36],[0.706077449705658,0.703430247591779,0.700496557254355,0.697258215559125,0.693717161567986,0.689824604790395,0.685557143780414,0.680888750137343,0.675794864016989,0.6702692042567,0.664280987762716,0.657801597332073,0.650807412153008,0.643275022725531,0.635195360381766,0.62655287715934,0.617327323505564,0.607507898870021,0.597087858286795,0.586063014547865,0.574431437124331,0.562203061568757,0.549389855065007,0.536009576910122,0.522085992888927,0.507648973305162,0.492734459423145,0.477384287700965,0.46164586681876,0.445571721235756,0.429218690207131,0.412647831239387,0.395921824841161,0.379107762934796,0.362272712824462,0.345483831650209,0.328807372215994,0.312307719055367,0.296046479879841,0.280081655961476,0.26446691294727,0.249250970448866,0.234477124201058,0.220182908861838,0.20639990324484,0.193153673653968,0.180463845635208,0.168344290252604,0.156803408086016,0.145844492514237,0.135466153354548,0.125662782389201,0.116425043511626,0.107740371964297,0.0995934692250543,0.0919667823602378,0.0848409589583944,0.0781952709730397,0.0720080028513533,0.066256801149935,0.0609189844059247,0.0559718133301342,0.0513927224231764,0.0471595092754717,0.0432505194230886,0.039644736415945,0.0363219113027158,0.0332626303175979,0.0304483683393607,0.027861523132039,0.0254854331756456,0.0233043816908055,0.0213035892335957,0.0194691969984717,0.0177882427276341,0.0162486308918009,0.0148390985853547,0.0135491783717696,0.0123691591252758,0.0112900457429703,0.0103035184482763,0.00940189227140855,0.00857807717448676,0.00782553918702825,0.0071382628304283,0.00651071433421108,0.00593781016730753,0.00541487956724159,0.00493763700329509,0.00450215211363538,0.0041048223479984,0.00374234736217256,0.00341170511596163,0.00311012961421417,0.00283509022185184,0.00258427247799794,0.00235556033077415,0.00214701971264914,0.00195688339548347,9.31691573210339e-37],[0.469790872875932,0.466981195264038,0.46403095826338,0.460919113852276,0.457619404871386,0.454213351786087,0.450622334400755,0.446830024814297,0.442814081428501,0.438549506157646,0.434078571211939,0.429346889948886,0.424339139953676,0.419037516944616,0.41342763594928,0.407512920787567,0.401270031872936,0.394690330852508,0.387767445542125,0.380490555129919,0.372871207587665,0.364905636876316,0.356598926399024,0.347959653698111,0.339000006065584,0.32973582925313,0.320186604542094,0.310375352254664,0.300328462482144,0.290075454609342,0.279648884274751,0.269074947835606,0.258406964742321,0.247675849636118,0.236921097845597,0.226182702395044,0.215500604161196,0.204914137536825,0.194461484154672,0.184179149034162,0.174101474480758,0.164260206596663,0.154684127277282,0.145398761428832,0.136426165393434,0.127784798718853,0.119489477827205,0.111551407035426,0.103978279862953,0.0967744416649585,0.089941103351298,0.0834765952675798,0.0773766501811209,0.0716347046677462,0.0662422089544109,0.0611889363435115,0.0564632846271376,0.0520525632946589,0.0479432617550118,0.0441212951603472,0.0405722256724035,0.0372814581176119,0.0342344099094245,0.0314166558980557,0.0288140492307658,0.0264128200032425,0.0241996537250234,0.0221617511876118,0.0202868722685986,0.0185633656134197,0.0169801861975348,0.0155269026474401,0.0141936960481855,0.0129713517996858,0.0118512459133358,0.0108253269712589,0.00988609480800868,0.00902657682216598,0.00824030268517436,0.00752127808807414,0.0068639580538698,0.0062632202438877,0.00571433860003334,0.00521295759046953,0.00475506726288488,0.00433697673520115,0.0039553019675039,0.00360692686185512,0.0032889948986022,0.00299888638203872,0.00273420019862217,0.0024927367398191,0.00227248195901685,0.00207159252334183,0.00188838201506183,0.00172130813308004,0.00156896084247408,0.00143005141877336,0.00130340240076682,6.20530867441762e-37],[0.224810026012268,0.225822973210475,0.226743453220302,0.227579039782647,0.228341955465573,0.228898027038248,0.229328193206956,0.229628742626153,0.229803238888584,0.229851340075665,0.22971176029313,0.229418032420034,0.228963524319746,0.228343513640723,0.227531133318551,0.226551959052886,0.225385224762491,0.224022345221941,0.222456154954673,0.220708984018545,0.218699600899977,0.21645564533946,0.213968301057178,0.211229424504512,0.208231894336989,0.204969990101795,0.201439785024866,0.197639534022278,0.193570035559005,0.189234951061999,0.184640687023096,0.179795550899776,0.174718962991882,0.169422879974238,0.163927177418616,0.158254710310155,0.152430970093633,0.146483657014747,0.140442184657111,0.134337135149801,0.128199684583847,0.122061019004401,0.115951761732106,0.109901432280179,0.10393795543792,0.09808723613657,0.0923728117409568,0.0868155888474185,0.0814336669963434,0.0762422473574224,0.0712536207397588,0.0664772264115897,0.0619197712594606,0.0575853977500253,0.0534758888847303,0.0495908987211655,0.0459281979184305,0.0424839249906275,0.0392528353783077,0.0362285419489567,0.0334037420157817,0.0307704273461165,0.0283200748676967,0.0260437892474031,0.0239325706347821,0.0219772513963706,0.0201687449977133,0.0184980980154335,0.0169565640327764,0.0155356622930501,0.0142272228101135,0.0130234196209571,0.0119167938016534,0.0109002677696042,0.00996715227455231,0.00911114734820954,0.00832633834531088,0.00760718807318095,0.00694852587661534,0.00634553442279163,0.00579373481872901,0.00528897059237336,0.00482739097794061,0.00440543386650188,0.00401980871342666,0.0036674784370834,0.00334564795262661,0.00305174006072219,0.00278338579599996,0.00253840813091318,0.00231480796425401,0.00211075086301009,0.0019245545682257,0.00175467726278145,0.00159970658907519,0.00145834939701053,0.00132942219707411,0.00121184228926288,0.00110461957080205,5.26388924080661e-37],[0.427104208658998,0.425758876397542,0.424302363195057,0.42272539362693,0.421023772602786,0.419256680443644,0.417355535988732,0.415307796701338,0.41309850487179,0.41070752149402,0.408211000757647,0.405536051305996,0.40266916894691,0.399592141115332,0.396270382907924,0.392778549613019,0.389046612050294,0.385060972739867,0.380808884653245,0.376288254987351,0.3714483293869,0.36629904015848,0.360827940022056,0.355024021286301,0.348878181698857,0.342383702175282,0.335536717790959,0.328336662404996,0.320786667066152,0.312893906566271,0.304669665731685,0.296123621769576,0.287288316288014,0.278183320443214,0.268837759013519,0.259284566198359,0.249560020910913,0.239703172859134,0.229755176469093,0.219758556837159,0.209756437486936,0.199791762359901,0.189906543732022,0.180141164109352,0.170533754631927,0.161119666185168,0.151931043122157,0.142996503759433,0.134340926877713,0.125985339423361,0.117946897466461,0.110238950188374,0.102871175187389,0.0958497726301557,0.0891777056465211,0.0828549747587309,0.0768789149366753,0.0712445049630739,0.0659446800687146,0.0609706401647979,0.0563121473811462,0.0519578079571204,0.0478953347842903,0.0441117669423946,0.0405937770411544,0.0373277187378717,0.0342999008107144,0.0314967036606488,0.0289047022783258,0.0265107682017141,0.0243021521146963,0.0222665488777849,0.0203921468455388,0.0186676633253047,0.0170823679820154,0.0156260959062565,0.0142892519486239,0.0130628077919427,0.011938293092086,0.0109077818742633,0.00996387522969855,0.00909968122127435,0.00830879277858365,0.00758526424459161,0.00692358712870335,0.00631866553937482,0.00576579156610416,0.00526062121742765,0.00479915066150594,0.0043776933866534,0.00399285820351057,0.00364152822567539,0.00332084089004273,0.00302816905149524,0.00276110316504679,0.0025174345513483,0.00229513972797689,0.00209236577856749,0.00190741672690837,9.09847585853839e-37],[0.0101265144184768,0.0105206584655791,0.0109386427008881,0.0113821451719519,0.0118510718249012,0.012349326767767,0.0128772402280895,0.0134357572640803,0.0140254595580392,0.0146439258770899,0.0152980762511076,0.0159836292202799,0.0166992869870835,0.0174427459686395,0.0182087524089667,0.019001569671995,0.0198123239743159,0.020635721737284,0.0214657457900841,0.0222965023549801,0.0231157052638158,0.0239169865689092,0.0246907575195862,0.0254269160674042,0.0261150500895073,0.0267446739986266,0.0273054917411606,0.0277876767856711,0.0281821576806047,0.0284808988776011,0.0286771126970195,0.0287648407915089,0.028741989438797,0.0286062500670671,0.028357890170493,0.027999098526228,0.0275339264418547,0.0269681671564636,0.0263091790261876,0.0255656608019649,0.0247473896861336,0.0238649346939607,0.0229293588738225,0.0219519239751656,0.0209438101809418,0.0199158616876501,0.0188783664772731,0.0178408758783459,0.0168120667401331,0.0157996464758166,0.0148102990330253,0.0138496681209513,0.0129223728007613,0.0120320498195749,0.0111814167916057,0.0103723504329967,0.00960597445635349,0.00888275234028335,0.00820258092552291,0.00756488157911822,0.00696868645127115,0.00641271808032519,0.0058954612491165,0.00541522544095838,0.00497020481188763,0.00455851715020058,0.00417824918133183,0.00382748698095485,0.00350434181234519,0.003206970358022,0.00293359009957729,0.00268249056598942,0.0024520411181194,0.00224069587346875,0.00204699630675244,0.00186957199288853,0.00170713989273022,0.0015585025202566,0.00142254527412853,0.00129823316699855,0.00118460714278062,0.00108078013497576,0.00098593298766514,0.000899310334373758,0.000820216508087731,0.000748010917155752,0.000682106804723256,0.000621963304977159,0.000567084927149713,0.000517017372224156,0.000471344367688928,0.000429684696555752,0.000391689418751828,0.000357039280444876,0.000325442305012323,0.000296631558078388,0.000270363078196746,0.000246413964255496,0.000224580625984516,1.06866528784183e-37],[-0.025126200836529,-0.0244902979351204,-0.0238116385855271,-0.0230880920195546,-0.0223157509376279,-0.021498577288314,-0.0206320987692154,-0.0197151048392607,-0.0187466602293993,-0.0177243188898075,-0.0166530566394788,-0.0155305107329707,-0.0143577156441073,-0.0131364853858146,-0.0118679184474305,-0.0105584898157881,-0.00921016793133493,-0.00782779340981474,-0.00641679370418373,-0.00498259047764557,-0.00353558144775242,-0.00208276562568242,-0.000633178403109462,0.000803518806493818,0.00221717570402603,0.00359731890261137,0.0049333434130393,0.0062147252639853,0.00743124922900743,0.00857324190379713,0.00963186335253938,0.0105948662990989,0.0114645498795662,0.0122309682155513,0.0128902653244881,0.0134401621653371,0.0138799938410786,0.0142106958634827,0.0144347406761133,0.0145560285674449,0.0145797395570168,0.0145121545598015,0.0143604550646662,0.0141325107744928,0.013836664281981,0.0134815210358468,0.0130757517093664,0.0126279127227286,0.0121462891894253,0.0116387630479384,0.0111127076910696,0.010574909093468,0.0100315123230562,0.00948799144214543,0.00894914017377366,0.00841908032162094,0.00790128476429441,0.00739861186212393,0.00691334827664311,0.00644725746872932,0.00600163147294991,0.00557734391020169,0.00517490257169877,0.00479449677158922,0.0044360603428492,0.00409929249378302,0.00378371583869123,0.00348870854665634,0.0032135370410029,0.00295738465419283,0.00271937641220409,0.0024986001910858,0.00229412453263375,0.00210501343049447,0.00193033840665787,0.00176918819495902,0.00162067633609813,0.00148394697050493,0.00135817909332798,0.00124258951167531,0.0011364347193211,0.00103901187943315,0.00094965908218386,0.000867755021869831,0.000792718217676931,0.000724005615317894,0.000661112332776194,0.000603568262296792,0.000550937665208428,0.000502817037118293,0.000458833290167464,0.000418641931813426,0.00038192526551463,0.000348390632858604,0.00031776871181459,0.000289811881774137,0.00026429266275051,0.000241002233434029,0.000219749037895204,1.05063950675544e-37],[0.0404887018584416,0.0406562962437675,0.0408489401780713,0.0410661368211916,0.0413071261416032,0.0415839612502844,0.0418884368759919,0.0422201435295758,0.0425776497589375,0.0429593240373989,0.0433712648061779,0.043806574166633,0.0442629020107317,0.0447368223674543,0.0452238774740008,0.0457242729497238,0.0462298871614151,0.0467348476787341,0.0472323891421635,0.0477143659640563,0.0481730852130917,0.0485992837388251,0.0489831387571539,0.0493143174804741,0.0495821607624466,0.0497759066552141,0.0498849494318806,0.0498991267303582,0.0498090246577863,0.0496062903061505,0.0492839380575675,0.048836954961373,0.0482607167048512,0.0475547349622559,0.0467198741966343,0.0457592391097734,0.0446781403813054,0.0434839816322986,0.042186071510994,0.0407953692993417,0.0393241766191983,0.0377857911156045,0.0361941399431379,0.0345634112559739,0.0329077007612554,0.0312406880192626,0.029575353963294,0.0279237474915993,0.0262968053344164,0.0247042260232725,0.0231543958982584,0.021654362805109,0.0202098515043672,0.0188253138250478,0.0175040061849695,0.0162480871790913,0.0150587283992358,0.0139362323827188,0.012880152488164,0.0118894104711912,0.0109624085046208,0.0100971333000258,0.00929125080111066,0.00854219566856031,0.0078472235786271,0.00720350952216203,0.0066081763736774,0.00605834618562075,0.00555117631677522,0.00508388812977187,0.00465378917900318,0.00425828978427944,0.00389491483336482,0.00356131158786395,0.00325525418947611,0.00297464548316582,0.00271751669446211,0.00248202542270942,0.0022664523424088,0.00206919694176667,0.0018887725715737,0.00172380102852103,0.0015730068546998,0.00143521149882937,0.00130932745413622,0.00119435184879292,0.00108936339166074,0.000993512710676082,0.000906020101059047,0.000826169692312444,0.000753304695701045,0.000686822919284715,0.000626172552596515,0.000570848218602436,0.000520387287208495,0.000474366442091536,0.000432398490852898,0.000394129407287894,0.000359235608566186,1.71079918512494e-37],[0.0334720324139718,0.0336814306986924,0.0339107825654271,0.0341597054574518,0.0344249064614904,0.0347156201521921,0.0350265454135618,0.03535729013459,0.035706693666117,0.0360689809154169,0.0364519558467847,0.0368490143026061,0.0372578677635839,0.0376755313941445,0.0380923262183736,0.0385150796459382,0.038933613079197,0.0393424466574461,0.0397355032324775,0.0401088494668923,0.040450482683172,0.0407555667199105,0.0410164950567138,0.0412255798936237,0.0413752262614239,0.0414581226104359,0.0414674414426103,0.0413970425008609,0.0412416704143825,0.0409971319361538,0.0406606175680717,0.0402282868448056,0.0397044152069819,0.0390879976134946,0.0383816706564298,0.0375894093513333,0.0367164129890525,0.0357689514409811,0.0347541813455888,0.0336799439211207,0.0325545568178397,0.0313866113830243,0.0301847844076776,0.0289576705365229,0.0277136386835914,0.0264607134208734,0.0252064805727279,0.023958015148273,0.0227218291724141,0.0215038367950118,0.0203093341351781,0.0191429915458196,0.0180088562810535,0.0169103638582867,0.015850356688998,0.0148311087847103,0.0138543555175502,0.0129213275293484,0.0120327879489277,0.0111890721087075,0.0103901289655686,0.00963556344249284,0.00892467892896603,0.00825652629416421,0.0076299139321035,0.00704349697956737,0.00649577669180971,0.00598514493373334,0.00550991510823501,0.00506834996662194,0.00465868622918324,0.00427915603210749,0.00392800529129611,0.00360350913310338,0.00330398458664552,0.00302780076294191,0.00277338676424786,0.0025392375743239,0.00232391817900775,0.00212606615823889,0.00194439297742472,0.00177768418932213,0.00162479873879278,0.00148466754298224,0.00135629149954368,0.00123873958960127,0.00113114385509081,0.00103270005618259,0.000942661933130102,0.000860338718681508,0.00078509191724296,0.000716332109082794,0.000653515815032333,0.000596142449083088,0.000543751379447221,0.000495919112911961,0.000452256612553294,0.000412406754958998,0.000376041917354461,1.79812007189094e-37],[-0.0282233339181958,-0.0282402899491004,-0.0282351595593361,-0.0282081335826868,-0.0281631691922447,-0.028066824263922,-0.027937456998329,-0.0277734957617292,-0.0275748900001351,-0.0273465737952696,-0.027059088254857,-0.0267280061791033,-0.0263521260683473,-0.0259316359273403,-0.0254714322339777,-0.0249526413491086,-0.0243874678929584,-0.023776913217894,-0.0231227444269937,-0.0224284271884968,-0.0216967360437292,-0.020930968942553,-0.0201358871706113,-0.0193169228220434,-0.0184800874658422,-0.0176318515292099,-0.0167789975597794,-0.0159284524899451,-0.015087105943986,-0.0142616183361322,-0.0134582818517643,-0.0126808172728667,-0.0119387488722578,-0.0112333442906473,-0.0105676174715613,-0.00994356972746886,-0.00936218180679909,-0.00882344634165554,-0.00832643844886912,-0.00786942022350077,-0.00744997280943118,-0.00706514794672188,-0.00671162972448436,-0.00638589695276657,-0.0060843771609136,-0.00580358460412175,-0.0055402365675968,-0.00529134441204959,-0.00505427794096969,-0.00482680358413515,-0.0046070984501788,-0.00439374343929932,-0.00418569931688174,-0.00398226996181323,-0.00378305697735517,-0.00358790955901405,-0.0033968730289518,-0.00321013884389015,-0.00302799822984395,-0.00285080094844431,-0.0026789200995382,-0.00251272334328815,-0.00235255049942048,-0.00219869320884008,-0.00205140063274625,-0.00191084672842413,-0.00177714655224973,-0.00165035207773491,-0.00153045529777583,-0.00141739302741096,-0.00131105281978192,-0.0012112795045432,-0.00111788195372454,-0.00103063976972514,-0.000949309670402965,-0.000873631415401127,-0.00080333317541982,-0.000738136292447844,-0.000677759414921118,-0.00062192201860176,-0.000570347343030916,-0.000522764786066922,-0.000478911806562071,-0.000438535388809753,-0.000401393123006054,-0.000367254284175512,-0.000335898893461068,-0.000307120214509093,-0.000280723156798325,-0.000256524605077751,-0.000234353174839672,-0.000214048871172437,-0.000195462674960843,-0.000178456076704024,-0.000162900574893815,-0.000148677152946384,-0.000135675746096231,-0.000123794707423965,-0.000112940273592965,-5.43087643953671e-38],[0.0192869048774205,0.0189324751010186,0.0185496510656108,0.0181372504841474,0.0176910730977523,0.0172113730139041,0.0166987047340754,0.016152604905961,0.0155727738189347,0.0149553178125877,0.0143035104077381,0.0136171569240061,0.0128969702453274,0.0121442010517508,0.0113562154490895,0.010540448903663,0.00969670215552043,0.00882754895505058,0.00793588166031413,0.00702607063584434,0.00610126517939963,0.00516652986094118,0.00422691083886238,0.00328785965598215,0.00235517529578164,0.00143493075365033,0.000533384335980025,-0.000343123404209619,-0.00118828540122672,-0.00199595381447689,-0.00276027266814046,-0.0034739099035116,-0.00413598675084078,-0.00474024755803015,-0.00528326558060628,-0.00576247845900061,-0.00617624752660531,-0.00652388815396344,-0.00680566940589974,-0.00702278325975767,-0.00717728553513505,-0.00727201228426216,-0.00731047655107271,-0.00729675108786058,-0.0072353428496755,-0.00713106492932256,-0.00698891112242413,-0.0068139375983044,-0.0066111552760258,-0.00638543554357503,-0.00614143098639942,-0.00588351187495803,-0.00561571835156594,-0.00534172758931379,-0.00506483468712479,-0.00478794571586757,-0.00451358112907102,-0.00424388767798772,-0.00398065700000645,-0.00372534915624169,-0.00347911955450246,-0.00324284788666548,-0.00301716791727876,-0.0028024947337943,-0.0025990639877631,-0.0024069424972984,-0.00222606596826938,-0.00205625913368501,-0.00189725699158835,-0.00174872375423158,-0.00161026953608659,-0.00148146486541407,-0.00136185314505369,-0.00125096121528315,-0.00114830818744799,-0.00105341272386018,-0.000965798939252025,-0.000885001093648997,-0.000810567237415088,-0.000742061957689347,-0.000679068362487285,-0.000621189425178177,-0.000568048798468664,-0.000519291193859798,-0.000474582410099357,-0.000433608576177678,-0.000396077829449781,-0.000361716253398046,-0.000330269572897993,-0.00030150164924396,-0.000275193583222823,-0.000251142786380124,-0.000229162039965356,-0.000209078557001289,-0.000190733059515704,-0.000173978880124726,-0.000158681094792287,-0.000144715691643506,-0.000131968793329268,-6.31698034591604e-38],[0.0491585272335494,0.0491724450540668,0.0491965153734501,0.0492301056863431,0.0492750860597926,0.0493410514137587,0.0494189430460779,0.0495076818438537,0.0496053830035448,0.0497136463093761,0.0498342944692224,0.0499611363919506,0.0500915295402786,0.0502219937664405,0.0503551247659676,0.0504788141068012,0.0505914302892045,0.050687986826555,0.0507628069489672,0.0508045507048977,0.0508175265702089,0.0507895148795676,0.0507133366683714,0.050581671542454,0.050387210360072,0.0501228297400815,0.0497817840564638,0.0493579088398396,0.0488458278425724,0.0482411583664003,0.0475406458439048,0.0467429688908906,0.0458462648169064,0.0448527963144183,0.0437654007859484,0.0425885194483035,0.041328132653483,0.0399916427209706,0.0385877080777775,0.0371260351885337,0.0356171372860637,0.0340720709595091,0.0325021629020009,0.0309187393675803,0.0293328701222008,0.0277551370440659,0.026195435285737,0.0246628123428526,0.0231653477549048,0.0217100737176068,0.0203029347773248,0.0189487831056619,0.0176514046590116,0.016413570808821,0.0152371097427415,0.0141229920198167,0.0130714250357399,0.0120819517339661,0.0111535496051966,0.0102847267803506,0.00947361278188156,0.00871804221060833,0.00801563027967579,0.00736383672071012,0.00676003644814845,0.00620154828789202,0.00568569155592844,0.0052098155563972,0.00477132666506539,0.00436770942846062,0.00399654245491553,0.00365550984496301,0.00334240885959317,0.00305515446370738,0.0027917813148119,0.00255044369816521,0.00232941384248357,0.002127078987093,0.0019419375134122,0.00177259440153587,0.00161775622666485,0.00147622587007765,0.00134689708492317,0.00122874902787446,0.00112084084309663,0.00102230603482634,0.000932348738403358,0.000850236545558054,0.000775297417098312,0.00070691478388764,0.000644523337770789,0.00058760507463012,0.000535685587710228,0.00048833060608633,0.000445142770740177,0.000405758638982177,0.000369845906783849,0.000337100837847584,0.000307245895737675,1.46271207913641e-37],[-1.11241931330987,-1.10686040587115,-1.10081183096275,-1.0942400993942,-1.08711444166358,-1.07938481881683,-1.07101995092941,-1.06198154731113,-1.05223207308413,-1.04173858739497,-1.0304567221507,-1.01835456495474,-1.00539887642249,-0.991559487734681,-0.976812280198608,-0.961134484805273,-0.944509598824069,-0.926927635971094,-0.908385608540965,-0.888888153867994,-0.868449344908497,-0.847092681560978,-0.82485132095011,-0.801768660445349,-0.777898472741877,-0.753304810222878,-0.728061658467385,-0.702252327287928,-0.675968577805598,-0.649309504907323,-0.622379988978647,-0.59529481748534,-0.568155111869229,-0.541079933098862,-0.5141817485926,-0.48757040385821,-0.461351410614992,-0.435624373817409,-0.410481602811076,-0.386006941339099,-0.362274840300699,-0.339349686714501,-0.317285392468373,-0.296125237193944,-0.275901951112668,-0.256638016235929,-0.238346158217765,-0.221029996768464,-0.204684820017794,-0.189298447569877,-0.17485214805466,-0.161321579468348,-0.148677724153417,-0.136887794526834,-0.125916090272823,-0.115724792370305,-0.106274683780224,-0.0975257906918585,-0.0894379417970421,-0.0819712460576132,-0.075086491830679,-0.0687454720316056,-0.0629112412871759,-0.0575483114874502,-0.0526227953778214,-0.0481024974611024,-0.0439569724071718,-0.0401575449477241,-0.0366773037420409,-0.0334910732197149,-0.0305753681952308,-0.0279083354164828,-0.0254696856016742,-0.0232406189528106,-0.0212037466190247,-0.0193430101240947,-0.0176436003708009,-0.0160918774885767,-0.0146752924967996,-0.0133823115095236,-0.0122023430034034,-0.0111256685037996,-0.0101433769095167,-0.00924730256956232,-0.00842996714138464,-0.00768452445916657,-0.00700471293304882,-0.00638480333101423,-0.005819558601531,-0.00530419232923918,-0.00483433124225918,-0.00440598060571075,-0.00401549232328658,-0.00365953557072072,-0.00333506978925498,-0.00303931987310589,-0.00276975339197182,-0.00252405969739945,-0.00230013078711009,-1.09313365851583e-36],[-0.788859568848064,-0.795900627034372,-0.80228522425614,-0.808109587470273,-0.813724330602225,-0.817726549041973,-0.820994269449277,-0.823533764094348,-0.825418203968049,-0.827038203165159,-0.827100736066698,-0.826357368524629,-0.824801167585949,-0.822487861988229,-0.819876851833235,-0.815633779019859,-0.810492298139314,-0.80442748998956,-0.797378599701264,-0.788982091964667,-0.780125357720822,-0.770297913147204,-0.759496624108251,-0.747723191238955,-0.734984790105662,-0.721294679500864,-0.706672758374394,-0.691146048736669,-0.674749079168644,-0.657524164576545,-0.639521031759679,-0.620950895066676,-0.601549781499604,-0.581569013880224,-0.561088197511051,-0.540192629566387,-0.518972281239705,-0.497520626510347,-0.475933345877259,-0.454306939486831,-0.432737290320033,-0.411318223603225,-0.39014011188615,-0.369288575013719,-0.348843320009633,-0.328877158086519,-0.309455225640595,-0.290634424488192,-0.272463085013707,-0.254980845335266,-0.238218730748233,-0.222199410956902,-0.206937608063405,-0.19244062585584,-0.17870897038704,-0.165737032835986,-0.153513807832128,-0.142023623434976,-0.131246862462407,-0.121160658562655,-0.111739554094701,-0.102956110345357,-0.0947814637490007,-0.0871858964986846,-0.0801389663811128,-0.0736103928029756,-0.0675700126883929,-0.061988155970362,-0.0568358683786726,-0.0520850894713732,-0.0477087905005719,-0.0436810767542725,-0.0399772589196526,-0.0365738978079336,-0.0334488264996908,-0.0305811536396174,-0.0279512512537435,-0.02554073009751,-0.0233324051833144,-0.0213102537909178,-0.0194593679401826,-0.0177659030072104,-0.016217023894368,-0.0148008499226779,-0.0135063994012382,-0.0123235476331712,-0.0112429178713262,-0.0102559170434462,-0.00935462211004951,-0.00853174876623586,-0.00778060530751876,-0.00709504889852034,-0.00646944429423678,-0.0058986250201912,-0.00537785698357433,-0.00490280446100119,-0.00446949838852578,-0.00407430686495214,-0.00371390742068229,-1.77026192503291e-36],[-0.347743181222223,-0.355851916017531,-0.363474158002545,-0.370711105258556,-0.37789796943717,-0.383774236965146,-0.389153401104028,-0.39405570660643,-0.398561864294463,-0.403041436244336,-0.406343205213268,-0.409159379290001,-0.41149724073489,-0.413419409828387,-0.415358736740234,-0.416109625061004,-0.416347715783053,-0.416056787993044,-0.415185847072872,-0.413401211501201,-0.411516902678052,-0.409060011209749,-0.406019806074267,-0.402386573149365,-0.398152149268491,-0.393310498707337,-0.387858319918509,-0.381795664923601,-0.375126548057356,-0.367859545128028,-0.360007717417003,-0.351717673299533,-0.342738339791068,-0.333251436911135,-0.323294075238197,-0.312909154312588,-0.30214499752502,-0.291054805870081,-0.279695937345283,-0.268129027063721,-0.256416973219943,-0.244623824647417,-0.232813614723698,-0.221049191836441,-0.209391097415832,-0.197896538648831,-0.186618495260983,-0.175604989484307,-0.164898536915839,-0.154535784645004,-0.144547332744813,-0.134957726626538,-0.125785601203472,-0.117043953402003,-0.108740517208762,-0.100878214921411,-0.0934556592590188,-0.0864676831233844,-0.0799058767181515,-0.0737591150865654,-0.0680140626268955,-0.0626556445513417,-0.0576674783971456,-0.053032289628847,-0.0487321317572566,-0.044748880246093,-0.0410643386912209,-0.0376604894866495,-0.0345196652124593,-0.0316246853318434,-0.0289589620519849,-0.026506579203968,-0.0242523478650665,-0.0221818422234582,-0.0202814189042104,-0.0185382226628622,-0.0169401810293553,-0.0154759901653196,-0.0141350938921429,-0.0129076575625921,-0.0117845381889117,-0.0107572520071022,-0.00981794045070735,-0.0089593353270958,-0.00817472383336402,-0.00745792402407848,-0.00680320803528166,-0.00620533775182975,-0.00565948412638388,-0.00516121025321108,-0.00470644262931397,-0.00429144397205773,-0.00391278761485767,-0.00356733347569435,-0.00325220557249033,-0.00296477104367798,-0.00270262062072226,-0.00246355049117805,-0.00224554520999916,-1.06991700303955e-36],[0.246158235322039,0.238260749588685,0.230755734859659,0.223548806170323,0.216363983327617,0.210193953217749,0.204359726748625,0.198826303373766,0.193511389361327,0.188123376751921,0.183516112138131,0.179128925732951,0.174930344775942,0.170845713741741,0.16653985961025,0.162880710642187,0.159332132392967,0.155879061682981,0.152533131431856,0.149512336415844,0.146181773923694,0.142872137173628,0.13956784588133,0.136254751997087,0.132920281430751,0.129553565591796,0.126145557164409,0.122689123989813,0.119179115395096,0.115612393581788,0.111987663044865,0.108246796411679,0.104525012480478,0.100751077834729,0.0969315467198554,0.0930747174202803,0.0891904621325782,0.0852900317882947,0.0813858377831866,0.0774912106057707,0.0736201351762819,0.0697869646758534,0.0660061181142014,0.0622917706490513,0.0586575485749835,0.0551162422279695,0.0516795496268466,0.0483578617377032,0.045160097273123,0.0420935914530763,0.0391640396296441,0.0363754934764858,0.0337304048077559,0.031229710150687,0.0288729479894056,0.02665840009065,0.0245832484284598,0.0226437398244084,0.0208353513760478,0.0191529509215122,0.0175909480568739,0.0161434324794672,0.014804297594816,0.0135673549063802,0.0124263980985469,0.0113753234974519,0.0104081605784433,0.00951912861388958,0.00870267322135942,0.00795349253390259,0.00726655450594654,0.00663710679672736,0.00606068056208464,0.00553308935145314,0.00505042416287801,0.00460904556439269,0.00420557365175162,0.00383687648454911,0.0035000575275647,0.00319244252274911,0.00291156612961404,0.00265515859726324,0.00242113266885369,0.00220757086764263,0.00201271327162512,0.00183494406111738,0.0016727880734756,0.00152488816132826,0.00139000419745378,0.00126700152746765,0.00115484244782128,0.00105257828582441,0.000959342056872548,0.000874341670935629,0.000796853658503461,0.000726217385296246,0.000661829724900478,0.000603140158893436,0.000549646316234317,2.61289237362878e-37],[0.501043235705578,0.49362818095156,0.48655268103229,0.479729654201357,0.47290188786529,0.466927482285775,0.461208514832606,0.455707421797421,0.450347048936412,0.444853719532225,0.439968491748583,0.435191841969441,0.430486195442783,0.425777376926221,0.420744136816902,0.416160150116286,0.411530844137191,0.406828956608694,0.402049329703151,0.397375672223295,0.392221787345048,0.386869755407884,0.381290626297659,0.375457116639637,0.369344097598962,0.362929121353444,0.356192972068146,0.349120222516007,0.341699773904781,0.333925345302431,0.325795760173878,0.317279030028392,0.308470618936572,0.299335693815302,0.289896406860646,0.280180576166993,0.27022127205434,0.260056246510081,0.249727224177663,0.239279078930944,0.228758924643056,0.218215152512922,0.207696450005381,0.197250837526094,0.186924757861595,0.176762249972806,0.166804233174588,0.157087920652618,0.147646373432095,0.13850819810046,0.129697384461642,0.121233273339489,0.113130640224287,0.10539987745065,0.0980472560365291,0.0910752480236788,0.0844828908994768,0.0782661771854521,0.0724184542931464,0.0669308220440345,0.0617925176367472,0.0569912801739808,0.0525136890271327,0.0483454680695861,0.0444717797593844,0.0408774403179325,0.0375471509795322,0.0344656724253603,0.0316179751811264,0.028989362854398,0.0265655707074639,0.0243328421767773,0.0222779859579693,0.0203884162023077,0.0186521782389367,0.0170579620663656,0.0155951056620569,0.0142535899529909,0.0130240270822672,0.011897643404206,0.0108662584479994,0.00992226091102235,0.00905858257937243,0.00826867092595221,0.00754646100553193,0.00688634729119565,0.00628315497517926,0.00573211337449886,0.00522882841362516,0.00476925697528347,0.00434968213746029,0.00396668958582282,0.00361714525592262,0.00329817423126696,0.00300714090075182,0.00274163036118972,0.00249943103696532,0.00227851847853765,0.00207704029269272,9.90387705544796e-37],[1.28052251312341,1.27367102073425,1.26709591022273,1.26070357149627,1.25424915767569,1.24844434353274,1.24274994178162,1.23710571870202,1.23141482532284,1.22540922297221,1.21966891180883,1.21373331364339,1.20752477549438,1.20093114549327,1.19362212998331,1.18620573169782,1.17820206965359,1.16952799785008,1.1601191810233,1.15008376319904,1.13883095090114,1.12655726921171,1.1131825252721,1.09863290476754,1.08284302760865,1.06575807623661,1.04733592008631,1.0275491462848,1.00638689824653,0.983856400635262,0.959984149017267,0.934784719106874,0.908399662191749,0.880868900488794,0.852296755214092,0.822804365464735,0.792527458245217,0.76161366539618,0.73021949919126,0.698507114320714,0.666640990415434,0.634784666644254,0.603097649414131,0.571732597362032,0.540832866403011,0.510530473469902,0.480944512646758,0.452180033437602,0.424327369423044,0.397461887642858,0.371644115352242,0.346920191536945,0.323322585553604,0.300871024006835,0.27957356881505,0.259427793598677,0.240422011297568,0.222536512608429,0.205744781857563,0.190014663845126,0.17530946169571,0.161588951619539,0.148810305614722,0.136928909315069,0.125899124450065,0.115674867439908,0.106210186854,0.0974597075229698,0.0893790041226633,0.0819249019881395,0.0750557126521499,0.0687314116221012,0.0629137657158651,0.0575664169259038,0.0526549293208614,0.0481468049639856,0.0440114742609687,0.0402202655718063,0.0367463583517618,0.0335647235400232,0.030652054401256,0.0279866905514663,0.0255485374689642,0.0233189834050457,0.0212808152668612,0.0194181352604256,0.0177162760587922,0.0161617238410869,0.0147420377345718,0.0134457757170498,0.0122624231750264,0.0111823250524766,0.0101966216972447,0.00929718843990782,0.00847657888261689,0.00772797183071106,0.00704512176576843,0.00642231273344296,0.00585431549687397,2.79098789184168e-36],[2.46178416738152,2.42736757487912,2.39099484377139,2.35247272494835,2.31146739382645,2.26847810567865,2.2229538632706,2.17476402920798,2.12376764229678,2.06967225337789,2.01299589076714,1.95334190912846,1.89069324794611,1.82504240512907,1.75617895740217,1.68493502190617,1.61105808563565,1.53475629338209,1.45630776970244,1.3762426368105,1.29436179472837,1.21137600870183,1.12773140214402,1.04391001202596,0.960421576592787,0.877793748093713,0.796561005418481,0.717252643801441,0.640380306589349,0.566425576397463,0.495827705977868,0.428964139214525,0.366193970904585,0.307748463999565,0.253812917933164,0.204498418598791,0.159842563708333,0.119812615872056,0.0843108167069253,0.0531814857995022,0.0262194521537346,0.00317932830860393,-0.0162148610807291,-0.0322601482691287,-0.0452655783676876,-0.0555439615968573,-0.063404643552493,-0.0691474280178736,-0.073057706686491,-0.0754027814964802,-0.0764293106682924,-0.0763617698171481,-0.0754017941428051,-0.0737282551580675,-0.0714979235757772,-0.0688465764330619,-0.0658904188913138,-0.0627277072044672,-0.0594404772077666,-0.0560963008338993,-0.052750010492015,-0.0494453468905148,-0.0462164996046493,-0.0430895420285805,-0.0400836205943389,-0.0372122353444534,-0.0344841411930664,-0.0319042479067424,-0.0294743722859853,-0.027193880127079,-0.0250602291189616,-0.0230694238819278,-0.0212163939875449,-0.0194953051530694,-0.0178998129891161,-0.0164232677799098,-0.0150588778521997,-0.0137998381846155,-0.0126394300522182,-0.0115710967084961,-0.0105884993881147,-0.00968555727142517,-0.00885647448511207,-0.00809575671873368,-0.00739821960901676,-0.00675898994262294,-0.00617350577486319,-0.00563750448087373,-0.00514701691768873,-0.00469835478464515,-0.00428809768285182,-0.00391307933018399,-0.00357037333389439,-0.00325727883464494,-0.00297130626347538,-0.00271016339433053,-0.00247174182704081,-0.00225410399720111,-0.00205547078501629,-9.83428291328154e-37]],"plugins":{"Unzoom":"{}"}},"evals":["attrs.interactionModel","plugins.Unzoom"],"jsHooks":[]}</script> --- ## Ridge Regression in R Variable importance: highly correlated predictors (e.g., self-criticism, depression, emotion regulation) have similar importance, and all predictors (regardless of actual relationship to eating disorder severity) remain in the model. ```r ggplot(varImp(ridgefit, scale = FALSE)) ``` <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-26-1.png" width="100%" /> --- class: inverse, center, middle # Lasso Regression --- ## Lasso Regression Lasso stands for the . Similar to ridge regression, lasso regression adds an additional penalty term (in this case, the `\(L1\)` penalty) to the OLS error function: `$$SSE_{L1} = \sum\limits_{i = 1}^n(y_i - \hat{y_i})^2 + \lambda \sum\limits_{j = 1}^P \lvert \beta_j \rvert$$` where - `\(n\)`: number of observations - `\(y_1\)`: `\(i\)`th observed response - `\(\hat{y_i}\)`: `\(i\)`th predicted response - `\(P\)`: number of parameters - `\(\beta_j\)`: the `\(j\)`th parameter --- ## Lasso Regression .left-column[ <br /> <img src="data:image/png;base64,#lasso.png" width="100%" /> ] .right-column[ Here, we aim to minimize the **absolute value** of all parameters (rather than the square of parameters as with ridge). While this may seem like a small change, it has the effect of , whereas ridge only shrinks values **towards** zero. If `\(\lambda\)` is set high enough, all parameters will be shrunk to zero. Ridge and lasso also differ in their handling of **multicollinearity.** Whereas ridge tends to shrink coefficients of correlated predictors towards each other, lasso tends to **pick one and ignore the rest**. ] --- ## Lasso Regression in R **Set tuning grid and train model** ```r *lassogrid <- expand.grid(alpha = 1, lambda = seq(0, 1, 0.01)) set.seed(2021) lassofit <- train(EDsim_recipe, data = EDsim_train, method = 'glmnet', * tuneGrid = lassogrid, trControl = EDsim_control) ``` -- .scroll-output[ ```r lassofit ``` ``` ## glmnet ## ## 400 samples ## 18 predictor ## ## Recipe steps: normalize, dummy ## Resampling: Cross-Validated (10 fold, repeated 3 times) ## Summary of sample sizes: 360, 360, 360, 360, 360, 360, ... ## Resampling results across tuning parameters: ## ## lambda RMSE Rsquared MAE ## 0.00 0.9878097 0.8266817 0.8442848 ## 0.01 0.9852440 0.8276105 0.8461214 ## 0.02 0.9830170 0.8285856 0.8466662 ## 0.03 0.9812399 0.8297381 0.8466669 ## 0.04 0.9807786 0.8306284 0.8472703 ## 0.05 0.9821676 0.8310571 0.8489385 ## 0.06 0.9851791 0.8310911 0.8518176 ## 0.07 0.9896117 0.8308058 0.8559999 ## 0.08 0.9951556 0.8303026 0.8606798 ## 0.09 1.0014885 0.8297017 0.8656008 ## 0.10 1.0086020 0.8289906 0.8707447 ## 0.11 1.0164096 0.8281740 0.8761148 ## 0.12 1.0246919 0.8272733 0.8817958 ## 0.13 1.0336049 0.8262467 0.8877364 ## 0.14 1.0431211 0.8251117 0.8942391 ## 0.15 1.0531958 0.8238716 0.9012422 ## 0.16 1.0637871 0.8225245 0.9087482 ## 0.17 1.0748689 0.8210682 0.9165948 ## 0.18 1.0864034 0.8195086 0.9247130 ## 0.19 1.0983483 0.8178511 0.9331077 ## 0.20 1.1107027 0.8160791 0.9417821 ## 0.21 1.1234328 0.8142041 0.9507356 ## 0.22 1.1365444 0.8121978 0.9601019 ## 0.23 1.1500444 0.8100395 0.9700940 ## 0.24 1.1638556 0.8077685 0.9808253 ## 0.25 1.1780201 0.8053384 0.9917830 ## 0.26 1.1924327 0.8027965 1.0029119 ## 0.27 1.2071404 0.8001048 1.0143176 ## 0.28 1.2221288 0.7972500 1.0260454 ## 0.29 1.2372428 0.7943216 1.0379057 ## 0.30 1.2523603 0.7913224 1.0497460 ## 0.31 1.2673596 0.7883327 1.0614408 ## 0.32 1.2823145 0.7853594 1.0730010 ## 0.33 1.2970967 0.7824258 1.0843288 ## 0.34 1.3116289 0.7796291 1.0955547 ## 0.35 1.3261227 0.7768838 1.1067770 ## 0.36 1.3404944 0.7741699 1.1179302 ## 0.37 1.3547458 0.7715430 1.1290978 ## 0.38 1.3690306 0.7689095 1.1403344 ## 0.39 1.3832846 0.7662809 1.1515775 ## 0.40 1.3977333 0.7634864 1.1630289 ## 0.41 1.4123811 0.7605217 1.1745401 ## 0.42 1.4271118 0.7574537 1.1861183 ## 0.43 1.4418901 0.7542702 1.1979148 ## 0.44 1.4566776 0.7509886 1.2096484 ## 0.45 1.4714230 0.7476944 1.2212870 ## 0.46 1.4860298 0.7444904 1.2327459 ## 0.47 1.5005886 0.7412740 1.2441449 ## 0.48 1.5149331 0.7382401 1.2553816 ## 0.49 1.5291968 0.7352612 1.2666565 ## 0.50 1.5435288 0.7321810 1.2780182 ## 0.51 1.5579223 0.7289758 1.2896291 ## 0.52 1.5722173 0.7257792 1.3012314 ## 0.53 1.5864083 0.7226110 1.3127800 ## 0.54 1.6003728 0.7196467 1.3242282 ## 0.55 1.6141524 0.7168193 1.3355269 ## 0.56 1.6278883 0.7139814 1.3468077 ## 0.57 1.6414625 0.7112093 1.3578777 ## 0.58 1.6548053 0.7086612 1.3687928 ## 0.59 1.6680531 0.7062716 1.3796365 ## 0.60 1.6812931 0.7038875 1.3905274 ## 0.61 1.6946249 0.7013876 1.4015632 ## 0.62 1.7080475 0.6987813 1.4126280 ## 0.63 1.7215611 0.6960040 1.4237203 ## 0.64 1.7351826 0.6930176 1.4348470 ## 0.65 1.7489066 0.6898100 1.4460547 ## 0.66 1.7627390 0.6863585 1.4572900 ## 0.67 1.7766462 0.6826953 1.4685849 ## 0.68 1.7906523 0.6787402 1.4799605 ## 0.69 1.8047186 0.6745203 1.4913085 ## 0.70 1.8187896 0.6700972 1.5025902 ## 0.71 1.8328145 0.6655144 1.5137945 ## 0.72 1.8468299 0.6606981 1.5249959 ## 0.73 1.8608483 0.6556467 1.5362307 ## 0.74 1.8747396 0.6504797 1.5473564 ## 0.75 1.8885597 0.6451125 1.5583849 ## 0.76 1.9022325 0.6397327 1.5692931 ## 0.77 1.9158058 0.6342642 1.5800956 ## 0.78 1.9292807 0.6287003 1.5908054 ## 0.79 1.9427082 0.6229453 1.6015181 ## 0.80 1.9560007 0.6171119 1.6121990 ## 0.81 1.9691814 0.6110608 1.6228253 ## 0.82 1.9822378 0.6047712 1.6333027 ## 0.83 1.9950736 0.5985670 1.6435786 ## 0.84 2.0076583 0.5923942 1.6536212 ## 0.85 2.0200561 0.5862781 1.6635936 ## 0.86 2.0322669 0.5802945 1.6734135 ## 0.87 2.0442680 0.5745290 1.6830773 ## 0.88 2.0561842 0.5686140 1.6927053 ## 0.89 2.0679809 0.5626438 1.7022312 ## 0.90 2.0797204 0.5563955 1.7117122 ## 0.91 2.0910839 0.5510484 1.7209727 ## 0.92 2.1024036 0.5454799 1.7301837 ## 0.93 2.1136027 0.5400764 1.7393180 ## 0.94 2.1247082 0.5347650 1.7484207 ## 0.95 2.1357762 0.5292462 1.7574550 ## 0.96 2.1468236 0.5233503 1.7664841 ## 0.97 2.1579099 0.5167551 1.7755404 ## 0.98 2.1689761 0.5094688 1.7845523 ## 0.99 2.1800575 0.5011838 1.7935547 ## 1.00 2.1910478 0.4921547 1.8024530 ## ## Tuning parameter 'alpha' was held constant at a value of 1 ## RMSE was used to select the optimal model using ## the smallest value. ## The final values used for the model were alpha = 1 ## and lambda = 0.04. ``` ] --- ## Lasso Regression in R **Predict on held-out test set** ```r lassopred_test <- predict(lassofit, EDsim_test) postResample(pred = lassopred_test, obs = EDsim_test_baked$ED_severity) ``` ``` ## RMSE Rsquared MAE ## 0.9390276 0.8654581 0.8224741 ``` --- ## Lasso Regression in R As `\(\lambda\)` increases to a certain point there's a small drop in RMSE, but then RMSE starts increasing past a certain point. ```r ggplot(lassofit) ``` <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-32-1.png" width="100%" /> --- ## Lasso Regression in R All coefficients shrink towards 0 as `\(\lambda\)` increases, with some actually becomming 0. ```r coefpath(lassofit$finalModel) ``` <div id="coefpath_iC4oBJULWg8m" style="width:100%;height:252px;" class="dygraphs html-widget"></div> <script type="application/json" data-for="coefpath_iC4oBJULWg8m">{"x":{"attrs":{"axes":{"x":{"pixelsPerLabel":60},"y":[]},"labels":["lambda","emo_reg","depression","impulsivity","self_crit","anxiety","age","family_history","prior_treatment","perfectionism","day_temperature","rainfall","num_siblings","num_cellphones","time_news","time_tv","num_pets","treatment_length","race_X3","race_X4","race_X5","race_X6","race_X7","race_X8"],"retainDateWindow":false,"xlabel":"Log Lambda","ylabel":"Coefficients","legend":"onmouseover","labelsDivWidth":250,"labelsShowZeroValues":true,"labelsSeparateLines":false,"showRangeSelector":true,"rangeSelectorHeight":40,"rangeSelectorPlotFillColor":" #A7B1C4","rangeSelectorPlotStrokeColor":"#808FAB","interactionModel":"Dygraph.Interaction.defaultModel","highlightCircleSize":3,"highlightSeriesBackgroundAlpha":0.5,"highlightSeriesOpts":{"strokeWidth":3},"hideOverlayOnMouseOut":true},"annotations":[{"x":-7.52747485487388,"shortText":"emo_reg","text":"emo_reg","width":50,"attachAtBottom":false,"series":"emo_reg"},{"x":-7.52747485487388,"shortText":"depression","text":"depression","width":50,"attachAtBottom":false,"series":"depression"},{"x":-7.52747485487388,"shortText":"impulsivity","text":"impulsivity","width":50,"attachAtBottom":false,"series":"impulsivity"},{"x":-7.52747485487388,"shortText":"self_crit","text":"self_crit","width":50,"attachAtBottom":false,"series":"self_crit"},{"x":-7.52747485487388,"shortText":"anxiety","text":"anxiety","width":50,"attachAtBottom":false,"series":"anxiety"},{"x":-7.52747485487388,"shortText":"age","text":"age","width":50,"attachAtBottom":false,"series":"age"},{"x":-7.52747485487388,"shortText":"family_history","text":"family_history","width":50,"attachAtBottom":false,"series":"family_history"},{"x":-7.52747485487388,"shortText":"prior_treatment","text":"prior_treatment","width":50,"attachAtBottom":false,"series":"prior_treatment"},{"x":-7.52747485487388,"shortText":"perfectionism","text":"perfectionism","width":50,"attachAtBottom":false,"series":"perfectionism"},{"x":-7.52747485487388,"shortText":"day_temperature","text":"day_temperature","width":50,"attachAtBottom":false,"series":"day_temperature"},{"x":-7.52747485487388,"shortText":"rainfall","text":"rainfall","width":50,"attachAtBottom":false,"series":"rainfall"},{"x":-7.52747485487388,"shortText":"num_siblings","text":"num_siblings","width":50,"attachAtBottom":false,"series":"num_siblings"},{"x":-7.52747485487388,"shortText":"num_cellphones","text":"num_cellphones","width":50,"attachAtBottom":false,"series":"num_cellphones"},{"x":-7.52747485487388,"shortText":"time_news","text":"time_news","width":50,"attachAtBottom":false,"series":"time_news"},{"x":-7.52747485487388,"shortText":"time_tv","text":"time_tv","width":50,"attachAtBottom":false,"series":"time_tv"},{"x":-7.52747485487388,"shortText":"num_pets","text":"num_pets","width":50,"attachAtBottom":false,"series":"num_pets"},{"x":-7.52747485487388,"shortText":"treatment_length","text":"treatment_length","width":50,"attachAtBottom":false,"series":"treatment_length"},{"x":-7.52747485487388,"shortText":"race_X3","text":"race_X3","width":50,"attachAtBottom":false,"series":"race_X3"},{"x":-7.52747485487388,"shortText":"race_X4","text":"race_X4","width":50,"attachAtBottom":false,"series":"race_X4"},{"x":-7.52747485487388,"shortText":"race_X5","text":"race_X5","width":50,"attachAtBottom":false,"series":"race_X5"},{"x":-7.52747485487388,"shortText":"race_X6","text":"race_X6","width":50,"attachAtBottom":false,"series":"race_X6"},{"x":-7.52747485487388,"shortText":"race_X7","text":"race_X7","width":50,"attachAtBottom":false,"series":"race_X7"},{"x":-7.52747485487388,"shortText":"race_X8","text":"race_X8","width":50,"attachAtBottom":false,"series":"race_X8"}],"shadings":[],"events":[],"format":"numeric","data":[[-7.52747485487388,-7.43444111382946,-7.34140737278503,-7.2483736317406,-7.15533989069617,-7.06230614965174,-6.96927240860732,-6.87623866756289,-6.78320492651846,-6.69017118547403,-6.5971374444296,-6.50410370338517,-6.41106996234075,-6.31803622129632,-6.22500248025189,-6.13196873920746,-6.03893499816303,-5.94590125711861,-5.85286751607418,-5.75983377502975,-5.66680003398532,-5.57376629294089,-5.48073255189647,-5.38769881085204,-5.29466506980761,-5.20163132876318,-5.10859758771875,-5.01556384667432,-4.9225301056299,-4.82949636458547,-4.73646262354104,-4.64342888249661,-4.55039514145218,-4.45736140040776,-4.36432765936333,-4.2712939183189,-4.17826017727447,-4.08522643623004,-3.99219269518562,-3.89915895414119,-3.80612521309676,-3.71309147205233,-3.6200577310079,-3.52702398996347,-3.43399024891905,-3.34095650787462,-3.24792276683019,-3.15488902578576,-3.06185528474133,-2.96882154369691,-2.87578780265248,-2.78275406160805,-2.68972032056362,-2.59668657951919,-2.50365283847477,-2.41061909743034,-2.31758535638591,-2.22455161534148,-2.13151787429705,-2.03848413325263,-1.9454503922082,-1.85241665116377,-1.75938291011934,-1.66634916907491,-1.57331542803048,-1.48028168698606,-1.38724794594163,-1.2942142048972,-1.20118046385277,-1.10814672280834,-1.01511298176392,-0.922079240719488,-0.82904549967506,-0.736011758630632,-0.642978017586204,-0.549944276541776,-0.456910535497347,-0.363876794452919,-0.270843053408491,-0.177809312364063,-0.0847755713196351,0.008258169724793,0.101291910769221,0.194325651813649],[0.689588780337484,0.687193074544508,0.685129167566831,0.683486480391731,0.681871048887744,0.680228824077784,0.67832075704149,0.675469278524861,0.671609147576537,0.666035478968801,0.663729896205063,0.66089644262467,0.658100169269402,0.654869561723774,0.651085141305179,0.64718032532786,0.643289593835311,0.638344840921175,0.633240814136042,0.627505173274387,0.621018998011021,0.615116326024691,0.610587113281549,0.602416726701941,0.594686017449844,0.593693452893034,0.593072658630693,0.592378242859904,0.591628177694998,0.590799509204744,0.589881993117669,0.587991359053661,0.540020433791056,0.397266679910221,0.240892305158892,0.00228211314231006,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.675534622446354,0.673834688165343,0.672565383255764,0.671790709712022,0.671522785377745,0.67149047040895,0.671449770386604,0.671376989398719,0.671312061290787,0.671183550724643,0.670995736867541,0.670798828477126,0.670566625733343,0.670311115644997,0.670038857451536,0.669740028086587,0.669397777082439,0.669038226300694,0.668641457784478,0.668227722451095,0.667798400468335,0.667411047679968,0.667034520193338,0.666592705369644,0.666169314332339,0.665724461853692,0.665262070705098,0.664759048558141,0.664202888017192,0.663594370342915,0.662937097431314,0.662244180573979,0.610528589455801,0.453182658248476,0.280819585203664,0.0178167058024951,0.0110605281673362,0.00656905097108176,0.00231315594514816,0.000334327931521451,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-0.148459159873895,-0.147193948222343,-0.145797884038924,-0.144349316494494,-0.142381227605581,-0.140069761632133,-0.137343858786955,-0.133045176262097,-0.127026013258245,-0.117897251504378,-0.114104341782599,-0.109488605575053,-0.104836370898858,-0.0993874187883758,-0.0929932807188977,-0.0864143145074937,-0.079828188748906,-0.0714756708578033,-0.0628623267930729,-0.0532074371004551,-0.0423024907036402,-0.0324303660620232,-0.0256578409766894,-0.0127432015568694,-0.000762333107629931,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.091016003396684,0.0919677073568752,0.0924833273710803,0.0925447647197169,0.0920785263392711,0.0913032266217206,0.0903685161875894,0.0887926465250946,0.086435134935783,0.0826613956007156,0.0811246015614519,0.079255274089421,0.0773356677623083,0.0750310582509067,0.0722970020619758,0.069496868086005,0.0666986918361622,0.0631289195635536,0.0594583131073985,0.0553404603974708,0.0506709334067679,0.0463771454265139,0.0438300756979682,0.0387012749424283,0.0340155096326251,0.034148286203179,0.0346517590475178,0.0352137911827932,0.0358218893456574,0.0364932303243787,0.0372279133767092,0.0385106770082804,0.082671788831249,0.215616266332964,0.361246717779111,0.583727526034186,0.587483556223643,0.588965752213302,0.590507036347003,0.590938939743582,0.590059550036893,0.589023663774164,0.588085821480791,0.586260207827813,0.584621306494307,0.58262170946548,0.580527485572824,0.578280359392406,0.575265576054664,0.573206221053909,0.569864215965639,0.566835208568578,0.563305628697189,0.559648224955896,0.555455325556962,0.550884284553882,0.545968960660456,0.54053529632145,0.534156672897618,0.527155814578476,0.519479791040399,0.511070934224182,0.502042318888315,0.491810718539293,0.480510431615509,0.467127499539838,0.452500839986511,0.436416548996173,0.418519032889249,0.398190362551058,0.381202129310252,0.365136503171376,0.347506036220919,0.326812302493554,0.301791791617141,0.275624021343041,0.24842733533227,0.218578766356917,0.185516014537453,0.15005510448615,0.106936296891843,0.0550435728541181,0,0],[-0.00497391931491759,-0.00288628141887362,-0.00131229268037107,-0.000313950797227274,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0603500036555654,0.246216220819958,0.449817706168946,0.760757097422318,0.765386966188009,0.767223536124257,0.768664146040236,0.768585524480017,0.767710131719142,0.766504978569123,0.765262213701985,0.764321088536614,0.763132867959347,0.762017898660395,0.760735409103348,0.758916446032947,0.757288606955973,0.754669183991702,0.752524421124137,0.74973957191596,0.74681026683155,0.74346145206589,0.739896500038699,0.735965140949932,0.731582843040616,0.726830347621987,0.722050608395486,0.716805052717836,0.711043415978213,0.704710206419518,0.697636593140385,0.690102732356347,0.681762794824661,0.673095741903536,0.663545428584485,0.653083439771054,0.641512919789142,0.628941061311368,0.611074226406069,0.58936873153649,0.565550812339629,0.540295365467145,0.514097246430524,0.484534334119372,0.451095459125635,0.414396481922677,0.374308799288071,0.317681133241725,0.254310044833604,0.18439176895344,0.108027215550539,0],[0.800472364873189,0.800187677118941,0.799997860007569,0.799907986201448,0.799949571862546,0.800058986989901,0.800193746529349,0.800443404806862,0.800823805330492,0.80144477681063,0.80163692762007,0.801880976461088,0.802119666003457,0.802412516792848,0.80276879671037,0.803124313132243,0.80346457373123,0.803927609346478,0.804391202613477,0.804922103085449,0.805531432379934,0.806049353522249,0.806258267316712,0.80692356348176,0.807482402252918,0.807129884543554,0.806675372867782,0.80617616961868,0.80562864153933,0.80502757360503,0.804368541366931,0.803541096349742,0.79252956622617,0.760166660273302,0.724713837230689,0.670963230891199,0.668964165978438,0.667362077075965,0.665659720755307,0.664106339397496,0.662651758164813,0.661160327734353,0.65972156498426,0.658173419300373,0.656154579975406,0.653783393023739,0.651175501854338,0.648263209127156,0.645118384625359,0.641624673938318,0.63805373052672,0.634075151978057,0.629726096674902,0.624934610798975,0.619690917511117,0.613933372917809,0.607601549667023,0.600722110775026,0.593675207233804,0.585941265724261,0.57745274794635,0.56813557067464,0.557893027472793,0.546665575309318,0.534386987745715,0.521004700223116,0.506312757096788,0.490189868772656,0.471745281102814,0.450671781847682,0.42570619948478,0.396603070659283,0.364663085698833,0.329740129995431,0.291631329481313,0.249715647930198,0.20360503447281,0.152998710703707,0.0974810727620587,0.0338394291034789,0,0,0,0],[0.538691353824296,0.538291431105315,0.537906826126718,0.537550734882683,0.537140209844572,0.536691518894672,0.536177525679093,0.535442307255764,0.534494477228582,0.533174970736562,0.532586227257262,0.531884088209918,0.531183818284618,0.530383560706228,0.529454886848495,0.528488889884037,0.527510129391192,0.526294751715031,0.525030298598752,0.523600286850968,0.521990387840363,0.520457655172117,0.519205510663822,0.517090826268546,0.515045538066643,0.514326827987325,0.513638921782651,0.512878808427056,0.51204932372474,0.511136815616803,0.510132891731992,0.509144361954374,0.503924159844117,0.490624312559643,0.476053570044951,0.454281933395356,0.452921214302993,0.451620966556075,0.45016874902406,0.448524671844005,0.446691270427742,0.444708358800167,0.442648787932679,0.440313955110358,0.437666762867292,0.434717942578694,0.431482077233204,0.428216253099562,0.424631976367133,0.42063052131121,0.416100505235401,0.411130034316583,0.405672113166837,0.399685029210725,0.393111429835447,0.38589742389696,0.377975944415324,0.369382532847152,0.360677819769377,0.351124402167312,0.340639661669353,0.329132925307796,0.316507945851268,0.302655564548161,0.287591251422688,0.271521958112005,0.253887516152659,0.234533204507945,0.2135586623908,0.190823258600055,0.165184433918501,0.136484177389501,0.104988573816731,0.0694767792855676,0.0291188861358498,0,0,0,0,0,0,0,0,0],[0.165542611579454,0.165925064968602,0.166293288298277,0.166628111072455,0.167028413566492,0.167479236630635,0.168015679718027,0.168876117330966,0.170125921835699,0.172043615127695,0.172820995234137,0.173786187940182,0.174754106470368,0.175914380083371,0.177302479255562,0.178730035268561,0.180150897577558,0.181963828403025,0.1838297971234,0.185941800709784,0.188333038123625,0.19057401186454,0.192063095397102,0.194790400852782,0.197327666013828,0.197263742036052,0.197007506775864,0.196724181670821,0.196415184781801,0.196075178478776,0.195702977199562,0.195656533965546,0.20000672387007,0.213995416573356,0.229317863196591,0.252837778313626,0.252347419101499,0.251598344005378,0.250785311842395,0.250084364546629,0.249477963769868,0.248835569973894,0.24813519371746,0.247552699492399,0.24679155017373,0.245956941431181,0.245038476504337,0.243995367736142,0.242915112022436,0.241437275527264,0.239651610736974,0.237629235746341,0.235431285658457,0.232996147227075,0.23034204025265,0.227425879314596,0.224203896969265,0.220579125443829,0.215665001787119,0.210271790287886,0.20435229229979,0.197854968037893,0.190700636941821,0.182828538744987,0.174127403967856,0.164043023655292,0.15296862668949,0.140817042715543,0.126659719643273,0.110243130148551,0.0862942321924339,0.0553143184139972,0.0213131774002569,0,0,0,0,0,0,0,0,0,0,0],[0.538303297876028,0.537887529260133,0.537593090746546,0.537437633060751,0.537468575337204,0.537580941443163,0.537698160830342,0.537820891295527,0.537906666052118,0.53791512143612,0.537877943376674,0.537826728122848,0.537750068696727,0.537652397682873,0.53754505750079,0.537434791872761,0.537319345072205,0.537179151500232,0.537033740609571,0.536877924667186,0.536708085736923,0.536636548169586,0.536676407264448,0.536739211687464,0.536834702818962,0.53713016002174,0.537462943776006,0.537836668302017,0.538238992169734,0.53868409830573,0.539177235738572,0.539377005537584,0.525584625210618,0.480527009750028,0.431169165305094,0.355853643326215,0.3540024410582,0.352823740300893,0.351686077418305,0.351152569180709,0.351049937457108,0.350970539781163,0.350767109242721,0.350352729204498,0.349711709464469,0.348941653922529,0.348098703964644,0.347225308247574,0.346295405795904,0.345195583705457,0.344037557379048,0.342740704676132,0.34132901892977,0.339767323459138,0.338063252550923,0.336191232499601,0.334121149259349,0.331833182280954,0.329004741065638,0.325900546885171,0.322493649850325,0.31875478682295,0.314637660759305,0.310077603411415,0.305183425299477,0.299717922287737,0.293716366101325,0.287130709886529,0.27943269623093,0.27049054876576,0.255908460391062,0.235833232749057,0.213798844649652,0.18978376371537,0.163666481122321,0.132541792853664,0.0932004401167079,0.0500233530807985,0.00263437213025965,0,0,0,0,0],[0.00100937495232314,0.00104128409942941,0.0010670157729008,0.00108539289707911,0.0011149137456659,0.00114528721417908,0.00117710339193056,0.00122515385652722,0.0012564267614142,0.00126845990596048,0.00119257160668585,0.00110480472832985,0.00099347917065443,0.000866012851177475,0.000728959849193397,0.000579002260153605,0.000413942974222783,0.000232848470134812,3.37339473960436e-05,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-0.0298592645375477,-0.0297811999271761,-0.0296922481665687,-0.0295924329604479,-0.0294725195323755,-0.0293373275572482,-0.0291877829674693,-0.0290108100204848,-0.0288125905946854,-0.0285787190603212,-0.0283812134942464,-0.0281632676009964,-0.0279287286883911,-0.0276581926879152,-0.0273458395956695,-0.0270058445272919,-0.026636237620653,-0.0262239773740936,-0.0257749742486292,-0.0252978030570576,-0.0247761657699034,-0.0241566993268072,-0.0235541115237425,-0.0228534806032789,-0.0221061858257678,-0.021356158814993,-0.0205398526523285,-0.0196426884624029,-0.0186592243899225,-0.0175793425397356,-0.0163940108888174,-0.0151779888778289,-0.0140548586241187,-0.0136857392458253,-0.0132784725882655,-0.0133083328817128,-0.0114517532998851,-0.00939504834152697,-0.00715379936563839,-0.00471835218867339,-0.00205096624941615,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.0442533647609414,0.0440976270680396,0.043939089596243,0.0437792072930732,0.04358259777686,0.0433664040987108,0.0431291424888704,0.0428427466019609,0.0425423093574827,0.0422174706889734,0.0420125029467458,0.0417879567043613,0.0415626595383215,0.0413279333488892,0.0410733358773728,0.0407969906231582,0.0404997005820629,0.0401639031971112,0.0397997528899246,0.0393807887595183,0.0389149555840609,0.0384188502876177,0.0378289560008276,0.0370445519103349,0.0362158821817915,0.0354022173694551,0.0345209318175099,0.0335484887657682,0.0324860643651297,0.031317864810031,0.0300341480563028,0.0288604504190377,0.0268412307019614,0.0243813562761098,0.0216843387100093,0.0181238022095095,0.0165685623550967,0.0148857461650401,0.0130307030453466,0.010973502599731,0.00871535761591111,0.00625795458764194,0.00367515685537701,0.0010582339267517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.0341213444882743,0.0339746631706892,0.0338237692356691,0.0336694469120515,0.033488235292792,0.0332921208699395,0.0330788170242974,0.0328270180479398,0.0325804154494481,0.0323366976550913,0.032180351921677,0.0320162729041868,0.0318542189016602,0.0316821999349473,0.0314891966733994,0.0312765616210197,0.0310424272974962,0.0307871185441946,0.0305059900752568,0.0302059202928798,0.0298786050631873,0.0294565343469371,0.0289614512693329,0.0283193164635864,0.0276338497601133,0.0268424870490903,0.0259796412363676,0.0250260033286547,0.023985541309252,0.0228408454264247,0.0215824866522697,0.0205757436375669,0.0188981515548114,0.0173061965524458,0.015560172016809,0.0133826046057294,0.0119937640034693,0.010491376734513,0.0088334579735309,0.00700311289584762,0.00500899318883722,0.00281948589960946,0.000346326688275582,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-0.01825540904367,-0.0182553228712291,-0.0182439246429016,-0.0182170399674185,-0.0181918790208995,-0.0181661653921792,-0.0181441530869686,-0.0181539932201737,-0.0182093779504349,-0.0183495360867052,-0.0183539071634743,-0.0183748826120188,-0.0183897697866022,-0.0184182211929325,-0.0184614852470955,-0.0184963530064921,-0.0185160553861615,-0.018570298739836,-0.0186139157633697,-0.0186632744008388,-0.0187250389031356,-0.0187196235337799,-0.0185880412790789,-0.0185742712567798,-0.0184989624513948,-0.0180356243047762,-0.0175036418712153,-0.0169181337225049,-0.0162770704669229,-0.0155728097086627,-0.0147989426843093,-0.0140849034259117,-0.0143850979352251,-0.0170236416968551,-0.0199120341672261,-0.0247743619745901,-0.0236661889859817,-0.022380809256971,-0.0209501197633644,-0.019315299289914,-0.0174716618153708,-0.0153652861895698,-0.0128221828387765,-0.00990240486594162,-0.00684694279323485,-0.00360517010373352,-4.85917028329872e-05,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.0137721842985146,0.0137806399155615,0.013773566731322,0.0137491485099633,0.0137054465285534,0.01364855121729,0.0135854567712375,0.013520755707416,0.0134418944215871,0.0133480522030107,0.0132135733867202,0.013063297248296,0.0128938278598051,0.0126975889594408,0.0124739335649442,0.0122284018564068,0.0119590900621031,0.0116634578929928,0.011339002681574,0.0109845694472382,0.010596310111606,0.0101492471750291,0.00970372479385265,0.00925984286614081,0.00876824949386012,0.00825180287461979,0.00768520294366475,0.00706512857158604,0.00638296488966527,0.00563503158452376,0.00481495737675355,0.00379443762406827,0.00397217399112833,0.00635600853691979,0.00896469581721259,0.0135315541325718,0.0119480502686516,0.010152094673021,0.00817133682734192,0.00598486842935993,0.00359391412902053,0.00105816551009991,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.059207248539715,0.0591778941015799,0.0591569390906952,0.0591461920951202,0.0591629166878283,0.0591855206807015,0.0592037238957943,0.0592143538414743,0.0591607932446418,0.0590141394040908,0.0588698186322774,0.058691247022634,0.0584849287523281,0.0582359130849978,0.0579489744211213,0.05764498971206,0.0573275502030087,0.0569507448606729,0.0565509737900482,0.0561074264133683,0.0556126040132129,0.0551504399037698,0.0548095586272255,0.0544346990498668,0.054051351946434,0.0539795928229135,0.0539153335865033,0.0538521602008691,0.0537760396672033,0.0536955745848513,0.0536094483734971,0.0530800021880817,0.0512165322729976,0.0450292430726881,0.0382499018626449,0.0282813500255979,0.0271003072752396,0.0258933331457146,0.0245850304348286,0.023181363647104,0.0216432735701764,0.0199722349767811,0.0182521630690796,0.0163220478451587,0.0141506561323286,0.0117235102376997,0.00906397956165653,0.00592126782022063,0.00244988926076492,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-1.17448480384475,-1.17444966736717,-1.1744163037479,-1.17438315811839,-1.17434698426228,-1.17430842577203,-1.17427014410193,-1.17425897868027,-1.17427541523018,-1.17435223924198,-1.17428498205399,-1.17421922859794,-1.17413799185116,-1.17406541611675,-1.17400446501952,-1.17392733006902,-1.17382863678313,-1.17374580134449,-1.17364245999515,-1.17352159362924,-1.17339486114947,-1.17325468773769,-1.17296247311664,-1.17276735007509,-1.17250108740848,-1.17194796332749,-1.17132219471539,-1.17063613432702,-1.16988251850738,-1.16905572809605,-1.1681489355579,-1.16704996079321,-1.16540580552134,-1.16284430761164,-1.16003594445161,-1.15629282100317,-1.15468699597442,-1.15294962604744,-1.15105436765096,-1.14900888200833,-1.14680040590885,-1.14438984337559,-1.14176701245449,-1.1387818612401,-1.1355159224813,-1.13194180550133,-1.12801673110168,-1.12388932001708,-1.11936605514793,-1.11432020970005,-1.10862885115576,-1.10237699900436,-1.09551657545112,-1.08798628518582,-1.07972273024066,-1.07065336860509,-1.06070252307913,-1.04995531859967,-1.03966783008543,-1.0283773248403,-1.01598587772973,-1.00238589651462,-0.987460767459809,-0.971099401386821,-0.95289159031968,-0.932303016562115,-0.909706250822665,-0.884906860267735,-0.857403797192708,-0.826888324975906,-0.793799599355057,-0.758437317624962,-0.719627840152231,-0.675981223247674,-0.626548082651789,-0.572057621921533,-0.511773787581314,-0.445612358329016,-0.373006393025898,-0.290836478096062,-0.199552210868363,-0.0983889084558275,0,0],[-0.223493815789978,-0.238142334425662,-0.252813925498383,-0.267279421512358,-0.285866581285104,-0.306122703386215,-0.328101423191894,-0.355568896228032,-0.381202728465774,-0.405805414140097,-0.4127376623263,-0.41922425013339,-0.423300040913852,-0.425050600332991,-0.425764055614245,-0.426512241895309,-0.427280960670557,-0.428216030393434,-0.429197442678735,-0.430299697150168,-0.431536656712177,-0.433614177453755,-0.448405585673996,-0.485552729344711,-0.522494528471996,-0.567827478432908,-0.616851551741469,-0.671698215954145,-0.730929531910887,-0.796372318967322,-0.868511041192939,-0.889437176342706,-0.983422212649356,-0.99855622295913,-1.01512887764642,-1.04182365775326,-1.03900915788761,-1.03545556581352,-1.03164968522041,-1.02695658680276,-1.02127863898871,-1.01511210295143,-1.00823814469201,-1.00041907083486,-0.99254833731375,-0.983966776731155,-0.974556160618555,-0.963603377536333,-0.951283456505606,-0.938480995215425,-0.923841571204763,-0.908097111581134,-0.890708650979185,-0.871740230088982,-0.850828959171639,-0.827895074400599,-0.80283438365,-0.775141013861019,-0.744194181973727,-0.710229909000115,-0.672956553150633,-0.632052910081856,-0.587275178286962,-0.538163191117309,-0.481632998791302,-0.412908980242916,-0.337507695060717,-0.254741416687171,-0.162983929121869,-0.0610038069705515,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.20269596441728,0.188005888407948,0.173254620105797,0.158655544943489,0.139923039591633,0.119505238342965,0.0973683933527158,0.0698753747069977,0.0443327168852561,0.0200938403000326,0.0128726044201017,0.00610717529781316,0.00168020478019281,0,0,0,0,0,0,0,0,0,-0.0124783647445984,-0.0488780871813933,-0.0852454637218387,-0.131099254794321,-0.180769549060174,-0.236323653643015,-0.296333037236587,-0.362629004462953,-0.435704301048131,-0.457880741794925,-0.550587292207917,-0.558626454568575,-0.567429260337815,-0.581754026100614,-0.579909854155436,-0.577577205004022,-0.575137569321298,-0.572212169960337,-0.568610791589861,-0.564936376870058,-0.561528855920364,-0.557683197901897,-0.553829688785631,-0.549472550019552,-0.54472206284946,-0.539257620303839,-0.532956434676108,-0.52667257795721,-0.519005737287138,-0.510935664843234,-0.501970142645826,-0.492245290674889,-0.481478344127888,-0.469677776307812,-0.456809989914671,-0.442481960581105,-0.425571847099496,-0.407012851549448,-0.3866474761171,-0.364302383425289,-0.339888122098189,-0.313019876283488,-0.281112695069325,-0.239196401680474,-0.193217223916989,-0.142742031485525,-0.0858510955491671,-0.0215139733283558,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.763632633136574,0.74922301091943,0.734737770886016,0.720373711703476,0.701943834458758,0.681868704478796,0.660132213669567,0.633256229453258,0.608540136280747,0.585519788272099,0.578799166299822,0.57264182221153,0.568818674532959,0.567498423770446,0.56749748925017,0.567434039494012,0.567278074342406,0.567262545923612,0.567167978746543,0.567058847053046,0.566971342171409,0.565657479877377,0.551428784694436,0.515446936362242,0.479242621080258,0.43202951457787,0.380770227375661,0.323475102026979,0.261552277281587,0.193157520961515,0.11777811415308,0.0931083734341465,0.00112130229548447,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.983893828153522,0.969691756034755,0.955393698850487,0.941182885818418,0.922943396257151,0.903079877846346,0.881598158574092,0.855170261493248,0.831097545634296,0.80908225493498,0.802635443041706,0.796834300187653,0.793340407110358,0.79248302328976,0.793127740126569,0.79372007601039,0.794204456982246,0.795030312499856,0.795792303584151,0.796640085589005,0.797641750058424,0.797447087275848,0.783982156940227,0.749243047432511,0.714125372029919,0.66656865926409,0.61482885647457,0.557009641648429,0.494508606934997,0.425480636988809,0.349407917511302,0.323871542932576,0.234242219586362,0.241601798497275,0.250319484659698,0.264545943279723,0.26197852237297,0.259039302153526,0.255742542951607,0.252177618048489,0.248409661293577,0.244299448543648,0.240180951026829,0.23635879546327,0.232082484828001,0.227449249725174,0.222349763585136,0.217080944023989,0.211427279638172,0.204844476288604,0.197715622893833,0.189745247619071,0.18104491298726,0.171446501031072,0.160953112141904,0.149429670531098,0.136747523495882,0.123014826091421,0.109280259225627,0.0942066610203742,0.0776620274531011,0.0595016503749529,0.0395239238550504,0.0176417534189441,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[1.72996231777587,1.71581838712914,1.7015525002746,1.68733408406233,1.66908689900906,1.64921746915736,1.62774953773863,1.60145933724142,1.57767534671178,1.55624902150395,1.54991785434255,1.54428777831261,1.54095603656337,1.54032960570007,1.54128350731812,1.54217274393597,1.54292546914963,1.54415032667404,1.54529825794452,1.54654139911879,1.54799502505741,1.54810926460547,1.53453763268885,1.50012184148786,1.46514137776901,1.4165688743894,1.36363783442968,1.30450787308942,1.24057139925573,1.16996662743745,1.0921606289451,1.06483609220928,0.976386060080981,0.991766403349774,1.00928860059654,1.0379068220217,1.03334560334307,1.02803004130939,1.02203768347955,1.01514482386881,1.00751041973035,0.998503156940962,0.986909562003465,0.974906057012818,0.962202948801152,0.948626671613333,0.933698566445973,0.918444693430287,0.901853799504538,0.883318274940745,0.863235920833987,0.841027735090256,0.816709862872251,0.789962455685437,0.76065590513944,0.728483770372054,0.693148233470821,0.654534297250216,0.613581921456184,0.568636827298122,0.519307619932097,0.465164595408519,0.405689422723378,0.34051803168799,0.271409416463712,0.203215194147863,0.128362110832008,0.0462180971589905,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[3.18781613609246,3.17273017147445,3.1574121966272,3.14200900572515,3.12243441540262,3.10111128655973,3.07808537356594,3.05028312314696,3.02514527534724,3.00274254647642,2.99439963658593,2.98666238820178,2.98097445070763,2.9778466529565,2.9761355752549,2.97401734811992,2.97134710121385,2.96903029793866,2.9661892708388,2.96291885540536,2.95945292889174,2.95375528646348,2.933001126338,2.89156067056318,2.84850802597781,2.7887370374095,2.72337633930277,2.65059764354721,2.57168835315207,2.48464792129266,2.38880443097525,2.34216310608614,2.23489521081082,2.23695395744578,2.23983723168736,2.25681198881763,2.22179625804419,2.18290658052674,2.14013325719177,2.09321050589898,2.04197181005673,1.98642855118391,1.92525857285844,1.85952149329685,1.78905727144477,1.71268114250312,1.62881448714703,1.53819443330753,1.4389708087656,1.32962639563089,1.2101413036722,1.0787355341545,0.93459573707264,0.776320550326831,0.602681491767548,0.412101516977163,0.20290359339561,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]],"plugins":{"Unzoom":"{}"}},"evals":["attrs.interactionModel","plugins.Unzoom"],"jsHooks":[]}</script> --- ## Lasso Regression in R Variable importance: Lasso has picked some highly correlated predictors to stay in the model (e.g., self-criticism) and set others (e.g., depression) to zero. Unimportant predictors have been removed from the model. ```r ggplot(varImp(lassofit, scale = FALSE)) ``` <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-35-1.png" width="100%" /> --- class: inverse, center, middle # Elastic Net Regression --- ## Elastic Net Regression Elastic net regression is a generalization of the lasso model, which combines both `\(L1\)` (lasso) and `\(L2\)` (ridge) penalty terms together in the error function: `$$SSE_{EN} = \sum\limits_{i = 1}^n(y_i - \hat{y_i})^2 + \lambda_1 \sum\limits_{j = 1}^P \beta_j^2 + \lambda_2 \sum\limits_{j = 1}^P \lvert \beta_j \rvert$$` where - `\(n\)`: number of observations - `\(y_1\)`: `\(i\)`th observed response - `\(\hat{y_i}\)`: `\(i\)`th predicted response - `\(P\)`: number of parameters - `\(\beta_j\)`: the `\(j\)`th parameter --- ## Elastic Net .left-column[ <br /> <img src="data:image/png;base64,#elasticnet.png" width="100%" /> ] .right-column[ The elastic net model allows for effective ridge-like regularization with lasso-like feature selection. In `caret`, we now tune two hyperparameters: `\(\lambda\)` and `\(\alpha\)`. As with ridge and lasso, `\(\lambda\)` controls the degree of regularization. The new `\(\alpha\)` hyperparameter is called the , and ranges from [0, 1]. At `\(\alpha = 0\)` ridge is performed and at `\(\alpha = 1\)` lasso is performed. Elastic net is particularly good at handling correlated predictors. ] --- ## Elastic Net in R **Set tuning grid and train model** ```r enfit <- train(EDsim_recipe, data = EDsim_train, method = 'glmnet', * tuneLength = 20, trControl = EDsim_control) ``` -- .scroll-output[ ```r enfit ``` ``` ## glmnet ## ## 400 samples ## 18 predictor ## ## Recipe steps: normalize, dummy ## Resampling: Cross-Validated (10 fold, repeated 3 times) ## Summary of sample sizes: 360, 360, 360, 360, 360, 360, ... ## Resampling results across tuning parameters: ## ## alpha lambda RMSE Rsquared MAE ## 0.1000000 0.0003766177 0.9879386 0.82498908 0.8447347 ## 0.1000000 0.0005839516 0.9879386 0.82498908 0.8447347 ## 0.1000000 0.0009054262 0.9879386 0.82498908 0.8447347 ## 0.1000000 0.0014038776 0.9879386 0.82498908 0.8447347 ## 0.1000000 0.0021767344 0.9879211 0.82499493 0.8447284 ## 0.1000000 0.0033750611 0.9876299 0.82508772 0.8446276 ## 0.1000000 0.0052330856 0.9871587 0.82524142 0.8445302 ## 0.1000000 0.0081139818 0.9866535 0.82540226 0.8445528 ## 0.1000000 0.0125808570 0.9864229 0.82547592 0.8448640 ## 0.1000000 0.0195068174 0.9860742 0.82559153 0.8453840 ## 0.1000000 0.0302456285 0.9853722 0.82584825 0.8459364 ## 0.1000000 0.0468963247 0.9842863 0.82629175 0.8465861 ## 0.1000000 0.0727134920 0.9832067 0.82685743 0.8478835 ## 0.1000000 0.1127434177 0.9834128 0.82733700 0.8508991 ## 0.1000000 0.1748104496 0.9862302 0.82782540 0.8567145 ## 0.1000000 0.2710463628 0.9953676 0.82817007 0.8673066 ## 0.1000000 0.4202616660 1.0184038 0.82789727 0.8887560 ## 0.1000000 0.6516223502 1.0674415 0.82606717 0.9332951 ## 0.1000000 1.0103507447 1.1582540 0.82084200 1.0066875 ## 0.1000000 1.5665647856 1.3065715 0.80845980 1.1196936 ## 0.1473684 0.0003766177 0.9878633 0.82504430 0.8445837 ## 0.1473684 0.0005839516 0.9878633 0.82504430 0.8445837 ## 0.1473684 0.0009054262 0.9878633 0.82504430 0.8445837 ## 0.1473684 0.0014038776 0.9878633 0.82504430 0.8445837 ## 0.1473684 0.0021767344 0.9877521 0.82507910 0.8445465 ## 0.1473684 0.0033750611 0.9873524 0.82520516 0.8444341 ## 0.1473684 0.0052330856 0.9868685 0.82535359 0.8444139 ## 0.1473684 0.0081139818 0.9864859 0.82546041 0.8445847 ## 0.1473684 0.0125808570 0.9864094 0.82547598 0.8450715 ## 0.1473684 0.0195068174 0.9859070 0.82564805 0.8455565 ## 0.1473684 0.0302456285 0.9849058 0.82601222 0.8459507 ## 0.1473684 0.0468963247 0.9835175 0.82656216 0.8465453 ## 0.1473684 0.0727134920 0.9825866 0.82711836 0.8482827 ## 0.1473684 0.1127434177 0.9829251 0.82770284 0.8517374 ## 0.1473684 0.1748104496 0.9861158 0.82847486 0.8578435 ## 0.1473684 0.2710463628 0.9973640 0.82906378 0.8697081 ## 0.1473684 0.4202616660 1.0268838 0.82857036 0.8961276 ## 0.1473684 0.6516223502 1.0899351 0.82566638 0.9494240 ## 0.1473684 1.0103507447 1.2051624 0.81792918 1.0379216 ## 0.1473684 1.5665647856 1.3912929 0.79841415 1.1816833 ## 0.1947368 0.0003766177 0.9880514 0.82498855 0.8447249 ## 0.1947368 0.0005839516 0.9880514 0.82498855 0.8447249 ## 0.1947368 0.0009054262 0.9880514 0.82498855 0.8447249 ## 0.1947368 0.0014038776 0.9880514 0.82498855 0.8447249 ## 0.1947368 0.0021767344 0.9878005 0.82506929 0.8446490 ## 0.1947368 0.0033750611 0.9873094 0.82522325 0.8445031 ## 0.1947368 0.0052330856 0.9867630 0.82538720 0.8444766 ## 0.1947368 0.0081139818 0.9864369 0.82547433 0.8446947 ## 0.1947368 0.0125808570 0.9863670 0.82548685 0.8452152 ## 0.1947368 0.0195068174 0.9856843 0.82572081 0.8456485 ## 0.1947368 0.0302456285 0.9844188 0.82618133 0.8459043 ## 0.1947368 0.0468963247 0.9828327 0.82681290 0.8466277 ## 0.1947368 0.0727134920 0.9820549 0.82736480 0.8486628 ## 0.1947368 0.1127434177 0.9824271 0.82813081 0.8522362 ## 0.1947368 0.1748104496 0.9868007 0.82895051 0.8592477 ## 0.1947368 0.2710463628 1.0017796 0.82928234 0.8735308 ## 0.1947368 0.4202616660 1.0394988 0.82809407 0.9056935 ## 0.1947368 0.6516223502 1.1170087 0.82387453 0.9673892 ## 0.1947368 1.0103507447 1.2583618 0.81222479 1.0756392 ## 0.1947368 1.5665647856 1.4816839 0.78149727 1.2492649 ## 0.2421053 0.0003766177 0.9879615 0.82499893 0.8446421 ## 0.2421053 0.0005839516 0.9879615 0.82499893 0.8446421 ## 0.2421053 0.0009054262 0.9879615 0.82499893 0.8446421 ## 0.2421053 0.0014038776 0.9879333 0.82500842 0.8446337 ## 0.2421053 0.0021767344 0.9875917 0.82512022 0.8445340 ## 0.2421053 0.0033750611 0.9871074 0.82527555 0.8444629 ## 0.2421053 0.0052330856 0.9865777 0.82543693 0.8444675 ## 0.2421053 0.0081139818 0.9863974 0.82548083 0.8447892 ## 0.2421053 0.0125808570 0.9862995 0.82550836 0.8453366 ## 0.2421053 0.0195068174 0.9853967 0.82581823 0.8456602 ## 0.2421053 0.0302456285 0.9838357 0.82637620 0.8458480 ## 0.2421053 0.0468963247 0.9823144 0.82699861 0.8468284 ## 0.2421053 0.0727134920 0.9814828 0.82765249 0.8489641 ## 0.2421053 0.1127434177 0.9823068 0.82847230 0.8526990 ## 0.2421053 0.1748104496 0.9883730 0.82920627 0.8610942 ## 0.2421053 0.2710463628 1.0077977 0.82911575 0.8780259 ## 0.2421053 0.4202616660 1.0546044 0.82691628 0.9162971 ## 0.2421053 0.6516223502 1.1484542 0.82046036 0.9878245 ## 0.2421053 1.0103507447 1.3168215 0.80292139 1.1184493 ## 0.2421053 1.5665647856 1.5726462 0.75726860 1.3176055 ## 0.2894737 0.0003766177 0.9877540 0.82505639 0.8444949 ## 0.2894737 0.0005839516 0.9877540 0.82505639 0.8444949 ## 0.2894737 0.0009054262 0.9877540 0.82505639 0.8444949 ## 0.2894737 0.0014038776 0.9877261 0.82506494 0.8444894 ## 0.2894737 0.0021767344 0.9873669 0.82518372 0.8443937 ## 0.2894737 0.0033750611 0.9868898 0.82534034 0.8443563 ## 0.2894737 0.0052330856 0.9864246 0.82548304 0.8444291 ## 0.2894737 0.0081139818 0.9863832 0.82548113 0.8448755 ## 0.2894737 0.0125808570 0.9861993 0.82553957 0.8454322 ## 0.2894737 0.0195068174 0.9850816 0.82592686 0.8456405 ## 0.2894737 0.0302456285 0.9832825 0.82656441 0.8458456 ## 0.2894737 0.0468963247 0.9818668 0.82716751 0.8470165 ## 0.2894737 0.0727134920 0.9811421 0.82788351 0.8492647 ## 0.2894737 0.1127434177 0.9825001 0.82874305 0.8534319 ## 0.2894737 0.1748104496 0.9906996 0.82929204 0.8631788 ## 0.2894737 0.2710463628 1.0153993 0.82856041 0.8836297 ## 0.2894737 0.4202616660 1.0714735 0.82522041 0.9276377 ## 0.2894737 0.6516223502 1.1830177 0.81580968 1.0116823 ## 0.2894737 1.0103507447 1.3781556 0.79057974 1.1637228 ## 0.2894737 1.5665647856 1.6545996 0.73872007 1.3801983 ## 0.3368421 0.0003766177 0.9877659 0.82504875 0.8445099 ## 0.3368421 0.0005839516 0.9877659 0.82504875 0.8445099 ## 0.3368421 0.0009054262 0.9877659 0.82504875 0.8445099 ## 0.3368421 0.0014038776 0.9876378 0.82509138 0.8444712 ## 0.3368421 0.0021767344 0.9872121 0.82523445 0.8443420 ## 0.3368421 0.0033750611 0.9867048 0.82539831 0.8442999 ## 0.3368421 0.0052330856 0.9863352 0.82550834 0.8444397 ## 0.3368421 0.0081139818 0.9863715 0.82548180 0.8449677 ## 0.3368421 0.0125808570 0.9860427 0.82558807 0.8454955 ## 0.3368421 0.0195068174 0.9846823 0.82605745 0.8455879 ## 0.3368421 0.0302456285 0.9827797 0.82674326 0.8458770 ## 0.3368421 0.0468963247 0.9814078 0.82735771 0.8471397 ## 0.3368421 0.0727134920 0.9809017 0.82809977 0.8495434 ## 0.3368421 0.1127434177 0.9830840 0.82891798 0.8544439 ## 0.3368421 0.1748104496 0.9937206 0.82921477 0.8655813 ## 0.3368421 0.2710463628 1.0241464 0.82773089 0.8898988 ## 0.3368421 0.4202616660 1.0901724 0.82298477 0.9399957 ## 0.3368421 0.6516223502 1.2197220 0.81011856 1.0385760 ## 0.3368421 1.0103507447 1.4415672 0.77461237 1.2112746 ## 0.3368421 1.5665647856 1.7349082 0.71934303 1.4440270 ## 0.3842105 0.0003766177 0.9877437 0.82506051 0.8444355 ## 0.3842105 0.0005839516 0.9877437 0.82506051 0.8444355 ## 0.3842105 0.0009054262 0.9877437 0.82506051 0.8444355 ## 0.3842105 0.0014038776 0.9874275 0.82516474 0.8443383 ## 0.3842105 0.0021767344 0.9870228 0.82530074 0.8442524 ## 0.3842105 0.0033750611 0.9865417 0.82545329 0.8442574 ## 0.3842105 0.0052330856 0.9862790 0.82552644 0.8444738 ## 0.3842105 0.0081139818 0.9863488 0.82548722 0.8450500 ## 0.3842105 0.0125808570 0.9858388 0.82565589 0.8455051 ## 0.3842105 0.0195068174 0.9842450 0.82620225 0.8455121 ## 0.3842105 0.0302456285 0.9823806 0.82687302 0.8459604 ## 0.3842105 0.0468963247 0.9811422 0.82749049 0.8473206 ## 0.3842105 0.0727134920 0.9808078 0.82827840 0.8498636 ## 0.3842105 0.1127434177 0.9839944 0.82901985 0.8555314 ## 0.3842105 0.1748104496 0.9975638 0.82893883 0.8683854 ## 0.3842105 0.2710463628 1.0336354 0.82670787 0.8965449 ## 0.3842105 0.4202616660 1.1103704 0.82027128 0.9531628 ## 0.3842105 0.6516223502 1.2582330 0.80328452 1.0675611 ## 0.3842105 1.0103507447 1.4996285 0.76063823 1.2551071 ## 0.3842105 1.5665647856 1.8151382 0.69654285 1.5077657 ## 0.4315789 0.0003766177 0.9874327 0.82516268 0.8443018 ## 0.4315789 0.0005839516 0.9874327 0.82516268 0.8443018 ## 0.4315789 0.0009054262 0.9874330 0.82516245 0.8443027 ## 0.4315789 0.0014038776 0.9872635 0.82521854 0.8442584 ## 0.4315789 0.0021767344 0.9868808 0.82534372 0.8442002 ## 0.4315789 0.0033750611 0.9863808 0.82550252 0.8442101 ## 0.4315789 0.0052330856 0.9862391 0.82553383 0.8445185 ## 0.4315789 0.0081139818 0.9863438 0.82548741 0.8451405 ## 0.4315789 0.0125808570 0.9856219 0.82573049 0.8454957 ## 0.4315789 0.0195068174 0.9838310 0.82633956 0.8454751 ## 0.4315789 0.0302456285 0.9820144 0.82699625 0.8460562 ## 0.4315789 0.0468963247 0.9809542 0.82760576 0.8475354 ## 0.4315789 0.0727134920 0.9808457 0.82842658 0.8502675 ## 0.4315789 0.1127434177 0.9852517 0.82904684 0.8567142 ## 0.4315789 0.1748104496 1.0020654 0.82848938 0.8714562 ## 0.4315789 0.2710463628 1.0438894 0.82544322 0.9034538 ## 0.4315789 0.4202616660 1.1319746 0.81707652 0.9674267 ## 0.4315789 0.6516223502 1.2978642 0.79521125 1.0969700 ## 0.4315789 1.0103507447 1.5544880 0.74909062 1.2969480 ## 0.4315789 1.5665647856 1.8926914 0.67712845 1.5695876 ## 0.4789474 0.0003766177 0.9871317 0.82525509 0.8442126 ## 0.4789474 0.0005839516 0.9871317 0.82525509 0.8442126 ## 0.4789474 0.0009054262 0.9871303 0.82525560 0.8442121 ## 0.4789474 0.0014038776 0.9869910 0.82530479 0.8441708 ## 0.4789474 0.0021767344 0.9866897 0.82540630 0.8441375 ## 0.4789474 0.0033750611 0.9862456 0.82554453 0.8441774 ## 0.4789474 0.0052330856 0.9862173 0.82553876 0.8445856 ## 0.4789474 0.0081139818 0.9863078 0.82549651 0.8452163 ## 0.4789474 0.0125808570 0.9853712 0.82581562 0.8454597 ## 0.4789474 0.0195068174 0.9834334 0.82647175 0.8454636 ## 0.4789474 0.0302456285 0.9816923 0.82711491 0.8461480 ## 0.4789474 0.0468963247 0.9807950 0.82771960 0.8477641 ## 0.4789474 0.0727134920 0.9810557 0.82852722 0.8507047 ## 0.4789474 0.1127434177 0.9868093 0.82900226 0.8580780 ## 0.4789474 0.1748104496 1.0070787 0.82792285 0.8748043 ## 0.4789474 0.2710463628 1.0549793 0.82394753 0.9108089 ## 0.4789474 0.4202616660 1.1542239 0.81361533 0.9826945 ## 0.4789474 0.6516223502 1.3385668 0.78556706 1.1268780 ## 0.4789474 1.0103507447 1.6096702 0.73728264 1.3396566 ## 0.4789474 1.5665647856 1.9742434 0.64570966 1.6337765 ## 0.5263158 0.0003766177 0.9869404 0.82532028 0.8441185 ## 0.5263158 0.0005839516 0.9869404 0.82532028 0.8441185 ## 0.5263158 0.0009054262 0.9869304 0.82532398 0.8441157 ## 0.5263158 0.0014038776 0.9867678 0.82538309 0.8440800 ## 0.5263158 0.0021767344 0.9865478 0.82545777 0.8440823 ## 0.5263158 0.0033750611 0.9861296 0.82558259 0.8441569 ## 0.5263158 0.0052330856 0.9861989 0.82554199 0.8446508 ## 0.5263158 0.0081139818 0.9862664 0.82550662 0.8452855 ## 0.5263158 0.0125808570 0.9850881 0.82590769 0.8454258 ## 0.5263158 0.0195068174 0.9830654 0.82659396 0.8454646 ## 0.5263158 0.0302456285 0.9815375 0.82718135 0.8462913 ## 0.5263158 0.0468963247 0.9807076 0.82781605 0.8479995 ## 0.5263158 0.0727134920 0.9814525 0.82858230 0.8512662 ## 0.5263158 0.1127434177 0.9886907 0.82888137 0.8595847 ## 0.5263158 0.1748104496 1.0125294 0.82725015 0.8784350 ## 0.5263158 0.2710463628 1.0667026 0.82227997 0.9183263 ## 0.5263158 0.4202616660 1.1765739 0.81007062 0.9989749 ## 0.5263158 0.6516223502 1.3790037 0.77538713 1.1570785 ## 0.5263158 1.0103507447 1.6672523 0.72261046 1.3854980 ## 0.5263158 1.5665647856 2.0546122 0.60064891 1.6969543 ## 0.5736842 0.0003766177 0.9868269 0.82535790 0.8441243 ## 0.5736842 0.0005839516 0.9868269 0.82535790 0.8441243 ## 0.5736842 0.0009054262 0.9868201 0.82536004 0.8441242 ## 0.5736842 0.0014038776 0.9866530 0.82542044 0.8440919 ## 0.5736842 0.0021767344 0.9864574 0.82548646 0.8441086 ## 0.5736842 0.0033750611 0.9860276 0.82561209 0.8441534 ## 0.5736842 0.0052330856 0.9861518 0.82555581 0.8446920 ## 0.5736842 0.0081139818 0.9861969 0.82552779 0.8453417 ## 0.5736842 0.0125808570 0.9848352 0.82598809 0.8454106 ## 0.5736842 0.0195068174 0.9827799 0.82668600 0.8455059 ## 0.5736842 0.0302456285 0.9814780 0.82721921 0.8464782 ## 0.5736842 0.0468963247 0.9807012 0.82788732 0.8482450 ## 0.5736842 0.0727134920 0.9819962 0.82861044 0.8518712 ## 0.5736842 0.1127434177 0.9909945 0.82864838 0.8612527 ## 0.5736842 0.1748104496 1.0183976 0.82646639 0.8824623 ## 0.5736842 0.2710463628 1.0783884 0.82071022 0.9255437 ## 0.5736842 0.4202616660 1.1999815 0.80589764 1.0164362 ## 0.5736842 0.6516223502 1.4171092 0.76664831 1.1858899 ## 0.5736842 1.0103507447 1.7239315 0.70751114 1.4309044 ## 0.5736842 1.5665647856 2.1255038 0.55975590 1.7524229 ## 0.6210526 0.0003766177 0.9869340 0.82531694 0.8439727 ## 0.6210526 0.0005839516 0.9869340 0.82531694 0.8439727 ## 0.6210526 0.0009054262 0.9868853 0.82533443 0.8439669 ## 0.6210526 0.0014038776 0.9865179 0.82546246 0.8438943 ## 0.6210526 0.0021767344 0.9861166 0.82559821 0.8438736 ## 0.6210526 0.0033750611 0.9859037 0.82565520 0.8441043 ## 0.6210526 0.0052330856 0.9861042 0.82556928 0.8447320 ## 0.6210526 0.0081139818 0.9860781 0.82556787 0.8453600 ## 0.6210526 0.0125808570 0.9845521 0.82607887 0.8453458 ## 0.6210526 0.0195068174 0.9825427 0.82675781 0.8455845 ## 0.6210526 0.0302456285 0.9814704 0.82724287 0.8466871 ## 0.6210526 0.0468963247 0.9807854 0.82793145 0.8485174 ## 0.6210526 0.0727134920 0.9826851 0.82860554 0.8524900 ## 0.6210526 0.1127434177 0.9936641 0.82830463 0.8631399 ## 0.6210526 0.1748104496 1.0241822 0.82574726 0.8864195 ## 0.6210526 0.2710463628 1.0904058 0.81903523 0.9328176 ## 0.6210526 0.4202616660 1.2246793 0.80091851 1.0354731 ## 0.6210526 0.6516223502 1.4545661 0.75841347 1.2149109 ## 0.6210526 1.0103507447 1.7781213 0.69519061 1.4742587 ## 0.6210526 1.5665647856 2.1909132 0.52277230 1.8045053 ## 0.6684211 0.0003766177 0.9867829 0.82535001 0.8439944 ## 0.6684211 0.0005839516 0.9867829 0.82535001 0.8439944 ## 0.6684211 0.0009054262 0.9867885 0.82534927 0.8440037 ## 0.6684211 0.0014038776 0.9863906 0.82548797 0.8439066 ## 0.6684211 0.0021767344 0.9859322 0.82565120 0.8438301 ## 0.6684211 0.0033750611 0.9857764 0.82569605 0.8440360 ## 0.6684211 0.0052330856 0.9859932 0.82560360 0.8447075 ## 0.6684211 0.0081139818 0.9859622 0.82560619 0.8453626 ## 0.6684211 0.0125808570 0.9842895 0.82615967 0.8453338 ## 0.6684211 0.0195068174 0.9823417 0.82682278 0.8456961 ## 0.6684211 0.0302456285 0.9815246 0.82724787 0.8469405 ## 0.6684211 0.0468963247 0.9809960 0.82793564 0.8488589 ## 0.6684211 0.0727134920 0.9835900 0.82853631 0.8532173 ## 0.6684211 0.1127434177 0.9963085 0.82799735 0.8649345 ## 0.6684211 0.1748104496 1.0298105 0.82508577 0.8900957 ## 0.6684211 0.2710463628 1.1031556 0.81706873 0.9411056 ## 0.6684211 0.4202616660 1.2503444 0.79521892 1.0550181 ## 0.6684211 0.6516223502 1.4921274 0.75012576 1.2440150 ## 0.6684211 1.0103507447 1.8332181 0.68119083 1.5181427 ## 0.6684211 1.5665647856 2.2558872 0.44646426 1.8564861 ## 0.7157895 0.0003766177 0.9868383 0.82539036 0.8439774 ## 0.7157895 0.0005839516 0.9868383 0.82539036 0.8439774 ## 0.7157895 0.0009054262 0.9867572 0.82541620 0.8439465 ## 0.7157895 0.0014038776 0.9862907 0.82556547 0.8438216 ## 0.7157895 0.0021767344 0.9857985 0.82571565 0.8437437 ## 0.7157895 0.0033750611 0.9856671 0.82574172 0.8439528 ## 0.7157895 0.0052330856 0.9858735 0.82564186 0.8446723 ## 0.7157895 0.0081139818 0.9857863 0.82566363 0.8453361 ## 0.7157895 0.0125808570 0.9840866 0.82622380 0.8453732 ## 0.7157895 0.0195068174 0.9822851 0.82684147 0.8458566 ## 0.7157895 0.0302456285 0.9816458 0.82723345 0.8472174 ## 0.7157895 0.0468963247 0.9813115 0.82791212 0.8492315 ## 0.7157895 0.0727134920 0.9843713 0.82852513 0.8538620 ## 0.7157895 0.1127434177 0.9987296 0.82779096 0.8665568 ## 0.7157895 0.1748104496 1.0360300 0.82421332 0.8942492 ## 0.7157895 0.2710463628 1.1164541 0.81489096 0.9500221 ## 0.7157895 0.4202616660 1.2768795 0.78878946 1.0747890 ## 0.7157895 0.6516223502 1.5299924 0.74133650 1.2730582 ## 0.7157895 1.0103507447 1.8897669 0.66188997 1.5631832 ## 0.7157895 1.5665647856 2.3089002 0.32270649 1.8971075 ## 0.7631579 0.0003766177 0.9870076 0.82534536 0.8440517 ## 0.7631579 0.0005839516 0.9870076 0.82534536 0.8440517 ## 0.7631579 0.0009054262 0.9868386 0.82540072 0.8439864 ## 0.7631579 0.0014038776 0.9862558 0.82558826 0.8438171 ## 0.7631579 0.0021767344 0.9856781 0.82576283 0.8436863 ## 0.7631579 0.0033750611 0.9856092 0.82576205 0.8439561 ## 0.7631579 0.0052330856 0.9857486 0.82568590 0.8446558 ## 0.7631579 0.0081139818 0.9856067 0.82572020 0.8452954 ## 0.7631579 0.0125808570 0.9839061 0.82627910 0.8454311 ## 0.7631579 0.0195068174 0.9823535 0.82682053 0.8460770 ## 0.7631579 0.0302456285 0.9818117 0.82720774 0.8474834 ## 0.7631579 0.0468963247 0.9814729 0.82795876 0.8494808 ## 0.7631579 0.0727134920 0.9848084 0.82865690 0.8542266 ## 0.7631579 0.1127434177 1.0014159 0.82750434 0.8683810 ## 0.7631579 0.1748104496 1.0425848 0.82324713 0.8985339 ## 0.7631579 0.2710463628 1.1302585 0.81252081 0.9599527 ## 0.7631579 0.4202616660 1.3036412 0.78199278 1.0944528 ## 0.7631579 0.6516223502 1.5688884 0.73120642 1.3032843 ## 0.7631579 1.0103507447 1.9463933 0.63712068 1.6082939 ## 0.7631579 1.5665647856 2.3431207 0.25361921 1.9218393 ## 0.8105263 0.0003766177 0.9871885 0.82526112 0.8442018 ## 0.8105263 0.0005839516 0.9871885 0.82526112 0.8442018 ## 0.8105263 0.0009054262 0.9869338 0.82534703 0.8441060 ## 0.8105263 0.0014038776 0.9863016 0.82555821 0.8439216 ## 0.8105263 0.0021767344 0.9856736 0.82576258 0.8437562 ## 0.8105263 0.0033750611 0.9855213 0.82579502 0.8439645 ## 0.8105263 0.0052330856 0.9856025 0.82573939 0.8445999 ## 0.8105263 0.0081139818 0.9853590 0.82579994 0.8452249 ## 0.8105263 0.0125808570 0.9837586 0.82632607 0.8455103 ## 0.8105263 0.0195068174 0.9825581 0.82675670 0.8463783 ## 0.8105263 0.0302456285 0.9819639 0.82719741 0.8477289 ## 0.8105263 0.0468963247 0.9812971 0.82812498 0.8494673 ## 0.8105263 0.0727134920 0.9853819 0.82875759 0.8547311 ## 0.8105263 0.1127434177 1.0043134 0.82716119 0.8703350 ## 0.8105263 0.1748104496 1.0493311 0.82224016 0.9028449 ## 0.8105263 0.2710463628 1.1444947 0.80995989 0.9705373 ## 0.8105263 0.4202616660 1.3293655 0.77555183 1.1139534 ## 0.8105263 0.6516223502 1.6074439 0.72050668 1.3336940 ## 0.8105263 1.0103507447 2.0017956 0.60807318 1.6525037 ## 0.8105263 1.5665647856 2.3507200 0.07296181 1.9268367 ## 0.8578947 0.0003766177 0.9872154 0.82524236 0.8441546 ## 0.8578947 0.0005839516 0.9872154 0.82524236 0.8441546 ## 0.8578947 0.0009054262 0.9868950 0.82535132 0.8440422 ## 0.8578947 0.0014038776 0.9862433 0.82557145 0.8438721 ## 0.8578947 0.0021767344 0.9856427 0.82576737 0.8437298 ## 0.8578947 0.0033750611 0.9854750 0.82580758 0.8439745 ## 0.8578947 0.0052330856 0.9855415 0.82575299 0.8445991 ## 0.8578947 0.0081139818 0.9851051 0.82587630 0.8452110 ## 0.8578947 0.0125808570 0.9836682 0.82634291 0.8456425 ## 0.8578947 0.0195068174 0.9828543 0.82666655 0.8467052 ## 0.8578947 0.0302456285 0.9817704 0.82730597 0.8477746 ## 0.8578947 0.0468963247 0.9810157 0.82833711 0.8492878 ## 0.8578947 0.0727134920 0.9861497 0.82880089 0.8553680 ## 0.8578947 0.1127434177 1.0073424 0.82679893 0.8723278 ## 0.8578947 0.1748104496 1.0562553 0.82121304 0.9071616 ## 0.8578947 0.2710463628 1.1591447 0.80715137 0.9815422 ## 0.8578947 0.4202616660 1.3539001 0.76984314 1.1324921 ## 0.8578947 0.6516223502 1.6438531 0.71138852 1.3628966 ## 0.8578947 1.0103507447 2.0531380 0.58238515 1.6933199 ## 0.8578947 1.5665647856 2.3508766 NaN 1.9269762 ## 0.9052632 0.0003766177 0.9871390 0.82528547 0.8442390 ## 0.9052632 0.0005839516 0.9871389 0.82528550 0.8442389 ## 0.9052632 0.0009054262 0.9867744 0.82540555 0.8441082 ## 0.9052632 0.0014038776 0.9860753 0.82563189 0.8438540 ## 0.9052632 0.0021767344 0.9854795 0.82582246 0.8436433 ## 0.9052632 0.0033750611 0.9853358 0.82585396 0.8439210 ## 0.9052632 0.0052330856 0.9854749 0.82577489 0.8446244 ## 0.9052632 0.0081139818 0.9849112 0.82593473 0.8452580 ## 0.9052632 0.0125808570 0.9836373 0.82634028 0.8458021 ## 0.9052632 0.0195068174 0.9831064 0.82660037 0.8469497 ## 0.9052632 0.0302456285 0.9814418 0.82745555 0.8476912 ## 0.9052632 0.0468963247 0.9807890 0.82853658 0.8491356 ## 0.9052632 0.0727134920 0.9871587 0.82876418 0.8561375 ## 0.9052632 0.1127434177 1.0105225 0.82640238 0.8743652 ## 0.9052632 0.1748104496 1.0633678 0.82013496 0.9114442 ## 0.9052632 0.2710463628 1.1741878 0.80409424 0.9928156 ## 0.9052632 0.4202616660 1.3778255 0.76453037 1.1509279 ## 0.9052632 0.6516223502 1.6786763 0.70401389 1.3909443 ## 0.9052632 1.0103507447 2.1029823 0.55690219 1.7328123 ## 0.9052632 1.5665647856 2.3508766 NaN 1.9269762 ## 0.9526316 0.0003766177 0.9872166 0.82524791 0.8443691 ## 0.9526316 0.0005839516 0.9872172 0.82524789 0.8443704 ## 0.9526316 0.0009054262 0.9867914 0.82538828 0.8442184 ## 0.9526316 0.0014038776 0.9860624 0.82562525 0.8439429 ## 0.9526316 0.0021767344 0.9854679 0.82582099 0.8437247 ## 0.9526316 0.0033750611 0.9853075 0.82585960 0.8439611 ## 0.9526316 0.0052330856 0.9853003 0.82583815 0.8446024 ## 0.9526316 0.0081139818 0.9845755 0.82604444 0.8451791 ## 0.9526316 0.0125808570 0.9836859 0.82631584 0.8460247 ## 0.9526316 0.0195068174 0.9829987 0.82666264 0.8469697 ## 0.9526316 0.0302456285 0.9810505 0.82762829 0.8475164 ## 0.9526316 0.0468963247 0.9806184 0.82872298 0.8490700 ## 0.9526316 0.0727134920 0.9883445 0.82867153 0.8570062 ## 0.9526316 0.1127434177 1.0138040 0.82597968 0.8764388 ## 0.9526316 0.1748104496 1.0706201 0.81902502 0.9157793 ## 0.9526316 0.2710463628 1.1894731 0.80087253 1.0042551 ## 0.9526316 0.4202616660 1.4023276 0.75882123 1.1698521 ## 0.9526316 0.6516223502 1.7143269 0.69597869 1.4197777 ## 0.9526316 1.0103507447 2.1517979 0.53173812 1.7721700 ## 0.9526316 1.5665647856 2.3508766 NaN 1.9269762 ## 1.0000000 0.0003766177 0.9870586 0.82530990 0.8442647 ## 1.0000000 0.0005839516 0.9870465 0.82531384 0.8442620 ## 1.0000000 0.0009054262 0.9865628 0.82547361 0.8440873 ## 1.0000000 0.0014038776 0.9858435 0.82570545 0.8438147 ## 1.0000000 0.0021767344 0.9853512 0.82586094 0.8436644 ## 1.0000000 0.0033750611 0.9852180 0.82588977 0.8439247 ## 1.0000000 0.0052330856 0.9852437 0.82586045 0.8446682 ## 1.0000000 0.0081139818 0.9842205 0.82616312 0.8451157 ## 1.0000000 0.0125808570 0.9836791 0.82629901 0.8462079 ## 1.0000000 0.0195068174 0.9827485 0.82676567 0.8469122 ## 1.0000000 0.0302456285 0.9806611 0.82780353 0.8473152 ## 1.0000000 0.0468963247 0.9805531 0.82887725 0.8490681 ## 1.0000000 0.0727134920 0.9896645 0.82854125 0.8579912 ## 1.0000000 0.1127434177 1.0171630 0.82553647 0.8785588 ## 1.0000000 0.1748104496 1.0780433 0.81787046 0.9205035 ## 1.0000000 0.2710463628 1.2051953 0.79735976 1.0159083 ## 1.0000000 0.4202616660 1.4271833 0.75266088 1.1890723 ## 1.0000000 0.6516223502 1.7511758 0.68654887 1.4494604 ## 1.0000000 1.0103507447 2.2014678 0.49193339 1.8124060 ## 1.0000000 1.5665647856 2.3508766 NaN 1.9269762 ## ## RMSE was used to select the optimal model using ## the smallest value. ## The final values used for the model were alpha = 1 ## and lambda = 0.04689632. ``` ] --- ## Elastic Net in R **Predict on held-out test set** ```r enpred_test <- predict(enfit, EDsim_test) postResample(pred = enpred_test, obs = EDsim_test_baked$ED_severity) ``` ``` ## RMSE Rsquared MAE ## 0.9417565 0.8654079 0.8268666 ``` --- ## Elastic Net in R ```r ggplot(enfit) ``` <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-41-1.png" width="100%" /> --- ## Elastic Net in R ```r coefpath(enfit$finalModel) ``` <div id="coefpath_lCUtO46Wyee0" style="width:100%;height:252px;" class="dygraphs html-widget"></div> <script type="application/json" data-for="coefpath_lCUtO46Wyee0">{"x":{"attrs":{"axes":{"x":{"pixelsPerLabel":60},"y":[]},"labels":["lambda","emo_reg","depression","impulsivity","self_crit","anxiety","age","family_history","prior_treatment","perfectionism","day_temperature","rainfall","num_siblings","num_cellphones","time_news","time_tv","num_pets","treatment_length","race_X3","race_X4","race_X5","race_X6","race_X7","race_X8"],"retainDateWindow":false,"xlabel":"Log Lambda","ylabel":"Coefficients","legend":"onmouseover","labelsDivWidth":250,"labelsShowZeroValues":true,"labelsSeparateLines":false,"showRangeSelector":true,"rangeSelectorHeight":40,"rangeSelectorPlotFillColor":" #A7B1C4","rangeSelectorPlotStrokeColor":"#808FAB","interactionModel":"Dygraph.Interaction.defaultModel","highlightCircleSize":3,"highlightSeriesBackgroundAlpha":0.5,"highlightSeriesOpts":{"strokeWidth":3},"hideOverlayOnMouseOut":true},"annotations":[{"x":-7.52747485487388,"shortText":"emo_reg","text":"emo_reg","width":50,"attachAtBottom":false,"series":"emo_reg"},{"x":-7.52747485487388,"shortText":"depression","text":"depression","width":50,"attachAtBottom":false,"series":"depression"},{"x":-7.52747485487388,"shortText":"impulsivity","text":"impulsivity","width":50,"attachAtBottom":false,"series":"impulsivity"},{"x":-7.52747485487388,"shortText":"self_crit","text":"self_crit","width":50,"attachAtBottom":false,"series":"self_crit"},{"x":-7.52747485487388,"shortText":"anxiety","text":"anxiety","width":50,"attachAtBottom":false,"series":"anxiety"},{"x":-7.52747485487388,"shortText":"age","text":"age","width":50,"attachAtBottom":false,"series":"age"},{"x":-7.52747485487388,"shortText":"family_history","text":"family_history","width":50,"attachAtBottom":false,"series":"family_history"},{"x":-7.52747485487388,"shortText":"prior_treatment","text":"prior_treatment","width":50,"attachAtBottom":false,"series":"prior_treatment"},{"x":-7.52747485487388,"shortText":"perfectionism","text":"perfectionism","width":50,"attachAtBottom":false,"series":"perfectionism"},{"x":-7.52747485487388,"shortText":"day_temperature","text":"day_temperature","width":50,"attachAtBottom":false,"series":"day_temperature"},{"x":-7.52747485487388,"shortText":"rainfall","text":"rainfall","width":50,"attachAtBottom":false,"series":"rainfall"},{"x":-7.52747485487388,"shortText":"num_siblings","text":"num_siblings","width":50,"attachAtBottom":false,"series":"num_siblings"},{"x":-7.52747485487388,"shortText":"num_cellphones","text":"num_cellphones","width":50,"attachAtBottom":false,"series":"num_cellphones"},{"x":-7.52747485487388,"shortText":"time_news","text":"time_news","width":50,"attachAtBottom":false,"series":"time_news"},{"x":-7.52747485487388,"shortText":"time_tv","text":"time_tv","width":50,"attachAtBottom":false,"series":"time_tv"},{"x":-7.52747485487388,"shortText":"num_pets","text":"num_pets","width":50,"attachAtBottom":false,"series":"num_pets"},{"x":-7.52747485487388,"shortText":"treatment_length","text":"treatment_length","width":50,"attachAtBottom":false,"series":"treatment_length"},{"x":-7.52747485487388,"shortText":"race_X3","text":"race_X3","width":50,"attachAtBottom":false,"series":"race_X3"},{"x":-7.52747485487388,"shortText":"race_X4","text":"race_X4","width":50,"attachAtBottom":false,"series":"race_X4"},{"x":-7.52747485487388,"shortText":"race_X5","text":"race_X5","width":50,"attachAtBottom":false,"series":"race_X5"},{"x":-7.52747485487388,"shortText":"race_X6","text":"race_X6","width":50,"attachAtBottom":false,"series":"race_X6"},{"x":-7.52747485487388,"shortText":"race_X7","text":"race_X7","width":50,"attachAtBottom":false,"series":"race_X7"},{"x":-7.52747485487388,"shortText":"race_X8","text":"race_X8","width":50,"attachAtBottom":false,"series":"race_X8"}],"shadings":[],"events":[],"format":"numeric","data":[[-7.52747485487388,-7.43444111382946,-7.34140737278503,-7.2483736317406,-7.15533989069617,-7.06230614965174,-6.96927240860732,-6.87623866756289,-6.78320492651846,-6.69017118547403,-6.5971374444296,-6.50410370338517,-6.41106996234075,-6.31803622129632,-6.22500248025189,-6.13196873920746,-6.03893499816303,-5.94590125711861,-5.85286751607418,-5.75983377502975,-5.66680003398532,-5.57376629294089,-5.48073255189647,-5.38769881085204,-5.29466506980761,-5.20163132876318,-5.10859758771875,-5.01556384667432,-4.9225301056299,-4.82949636458547,-4.73646262354104,-4.64342888249661,-4.55039514145218,-4.45736140040776,-4.36432765936333,-4.2712939183189,-4.17826017727447,-4.08522643623004,-3.99219269518562,-3.89915895414119,-3.80612521309676,-3.71309147205233,-3.6200577310079,-3.52702398996347,-3.43399024891905,-3.34095650787462,-3.24792276683019,-3.15488902578576,-3.06185528474133,-2.96882154369691,-2.87578780265248,-2.78275406160805,-2.68972032056362,-2.59668657951919,-2.50365283847477,-2.41061909743034,-2.31758535638591,-2.22455161534148,-2.13151787429705,-2.03848413325263,-1.9454503922082,-1.85241665116377,-1.75938291011934,-1.66634916907491,-1.57331542803048,-1.48028168698606,-1.38724794594163,-1.2942142048972,-1.20118046385277,-1.10814672280834,-1.01511298176392,-0.922079240719488,-0.82904549967506,-0.736011758630632,-0.642978017586204,-0.549944276541776,-0.456910535497347,-0.363876794452919,-0.270843053408491,-0.177809312364063,-0.0847755713196351,0.008258169724793,0.101291910769221,0.194325651813649],[0.689588780337484,0.687193074544508,0.685129167566831,0.683486480391731,0.681871048887744,0.680228824077784,0.67832075704149,0.675469278524861,0.671609147576537,0.666035478968801,0.663729896205063,0.66089644262467,0.658100169269402,0.654869561723774,0.651085141305179,0.64718032532786,0.643289593835311,0.638344840921175,0.633240814136042,0.627505173274387,0.621018998011021,0.615116326024691,0.610587113281549,0.602416726701941,0.594686017449844,0.593693452893034,0.593072658630693,0.592378242859904,0.591628177694998,0.590799509204744,0.589881993117669,0.587991359053661,0.540020433791056,0.397266679910221,0.240892305158892,0.00228211314231006,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.675534622446354,0.673834688165343,0.672565383255764,0.671790709712022,0.671522785377745,0.67149047040895,0.671449770386604,0.671376989398719,0.671312061290787,0.671183550724643,0.670995736867541,0.670798828477126,0.670566625733343,0.670311115644997,0.670038857451536,0.669740028086587,0.669397777082439,0.669038226300694,0.668641457784478,0.668227722451095,0.667798400468335,0.667411047679968,0.667034520193338,0.666592705369644,0.666169314332339,0.665724461853692,0.665262070705098,0.664759048558141,0.664202888017192,0.663594370342915,0.662937097431314,0.662244180573979,0.610528589455801,0.453182658248476,0.280819585203664,0.0178167058024951,0.0110605281673362,0.00656905097108176,0.00231315594514816,0.000334327931521451,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-0.148459159873895,-0.147193948222343,-0.145797884038924,-0.144349316494494,-0.142381227605581,-0.140069761632133,-0.137343858786955,-0.133045176262097,-0.127026013258245,-0.117897251504378,-0.114104341782599,-0.109488605575053,-0.104836370898858,-0.0993874187883758,-0.0929932807188977,-0.0864143145074937,-0.079828188748906,-0.0714756708578033,-0.0628623267930729,-0.0532074371004551,-0.0423024907036402,-0.0324303660620232,-0.0256578409766894,-0.0127432015568694,-0.000762333107629931,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.091016003396684,0.0919677073568752,0.0924833273710803,0.0925447647197169,0.0920785263392711,0.0913032266217206,0.0903685161875894,0.0887926465250946,0.086435134935783,0.0826613956007156,0.0811246015614519,0.079255274089421,0.0773356677623083,0.0750310582509067,0.0722970020619758,0.069496868086005,0.0666986918361622,0.0631289195635536,0.0594583131073985,0.0553404603974708,0.0506709334067679,0.0463771454265139,0.0438300756979682,0.0387012749424283,0.0340155096326251,0.034148286203179,0.0346517590475178,0.0352137911827932,0.0358218893456574,0.0364932303243787,0.0372279133767092,0.0385106770082804,0.082671788831249,0.215616266332964,0.361246717779111,0.583727526034186,0.587483556223643,0.588965752213302,0.590507036347003,0.590938939743582,0.590059550036893,0.589023663774164,0.588085821480791,0.586260207827813,0.584621306494307,0.58262170946548,0.580527485572824,0.578280359392406,0.575265576054664,0.573206221053909,0.569864215965639,0.566835208568578,0.563305628697189,0.559648224955896,0.555455325556962,0.550884284553882,0.545968960660456,0.54053529632145,0.534156672897618,0.527155814578476,0.519479791040399,0.511070934224182,0.502042318888315,0.491810718539293,0.480510431615509,0.467127499539838,0.452500839986511,0.436416548996173,0.418519032889249,0.398190362551058,0.381202129310252,0.365136503171376,0.347506036220919,0.326812302493554,0.301791791617141,0.275624021343041,0.24842733533227,0.218578766356917,0.185516014537453,0.15005510448615,0.106936296891843,0.0550435728541181,0,0],[-0.00497391931491759,-0.00288628141887362,-0.00131229268037107,-0.000313950797227274,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0603500036555654,0.246216220819958,0.449817706168946,0.760757097422318,0.765386966188009,0.767223536124257,0.768664146040236,0.768585524480017,0.767710131719142,0.766504978569123,0.765262213701985,0.764321088536614,0.763132867959347,0.762017898660395,0.760735409103348,0.758916446032947,0.757288606955973,0.754669183991702,0.752524421124137,0.74973957191596,0.74681026683155,0.74346145206589,0.739896500038699,0.735965140949932,0.731582843040616,0.726830347621987,0.722050608395486,0.716805052717836,0.711043415978213,0.704710206419518,0.697636593140385,0.690102732356347,0.681762794824661,0.673095741903536,0.663545428584485,0.653083439771054,0.641512919789142,0.628941061311368,0.611074226406069,0.58936873153649,0.565550812339629,0.540295365467145,0.514097246430524,0.484534334119372,0.451095459125635,0.414396481922677,0.374308799288071,0.317681133241725,0.254310044833604,0.18439176895344,0.108027215550539,0],[0.800472364873189,0.800187677118941,0.799997860007569,0.799907986201448,0.799949571862546,0.800058986989901,0.800193746529349,0.800443404806862,0.800823805330492,0.80144477681063,0.80163692762007,0.801880976461088,0.802119666003457,0.802412516792848,0.80276879671037,0.803124313132243,0.80346457373123,0.803927609346478,0.804391202613477,0.804922103085449,0.805531432379934,0.806049353522249,0.806258267316712,0.80692356348176,0.807482402252918,0.807129884543554,0.806675372867782,0.80617616961868,0.80562864153933,0.80502757360503,0.804368541366931,0.803541096349742,0.79252956622617,0.760166660273302,0.724713837230689,0.670963230891199,0.668964165978438,0.667362077075965,0.665659720755307,0.664106339397496,0.662651758164813,0.661160327734353,0.65972156498426,0.658173419300373,0.656154579975406,0.653783393023739,0.651175501854338,0.648263209127156,0.645118384625359,0.641624673938318,0.63805373052672,0.634075151978057,0.629726096674902,0.624934610798975,0.619690917511117,0.613933372917809,0.607601549667023,0.600722110775026,0.593675207233804,0.585941265724261,0.57745274794635,0.56813557067464,0.557893027472793,0.546665575309318,0.534386987745715,0.521004700223116,0.506312757096788,0.490189868772656,0.471745281102814,0.450671781847682,0.42570619948478,0.396603070659283,0.364663085698833,0.329740129995431,0.291631329481313,0.249715647930198,0.20360503447281,0.152998710703707,0.0974810727620587,0.0338394291034789,0,0,0,0],[0.538691353824296,0.538291431105315,0.537906826126718,0.537550734882683,0.537140209844572,0.536691518894672,0.536177525679093,0.535442307255764,0.534494477228582,0.533174970736562,0.532586227257262,0.531884088209918,0.531183818284618,0.530383560706228,0.529454886848495,0.528488889884037,0.527510129391192,0.526294751715031,0.525030298598752,0.523600286850968,0.521990387840363,0.520457655172117,0.519205510663822,0.517090826268546,0.515045538066643,0.514326827987325,0.513638921782651,0.512878808427056,0.51204932372474,0.511136815616803,0.510132891731992,0.509144361954374,0.503924159844117,0.490624312559643,0.476053570044951,0.454281933395356,0.452921214302993,0.451620966556075,0.45016874902406,0.448524671844005,0.446691270427742,0.444708358800167,0.442648787932679,0.440313955110358,0.437666762867292,0.434717942578694,0.431482077233204,0.428216253099562,0.424631976367133,0.42063052131121,0.416100505235401,0.411130034316583,0.405672113166837,0.399685029210725,0.393111429835447,0.38589742389696,0.377975944415324,0.369382532847152,0.360677819769377,0.351124402167312,0.340639661669353,0.329132925307796,0.316507945851268,0.302655564548161,0.287591251422688,0.271521958112005,0.253887516152659,0.234533204507945,0.2135586623908,0.190823258600055,0.165184433918501,0.136484177389501,0.104988573816731,0.0694767792855676,0.0291188861358498,0,0,0,0,0,0,0,0,0],[0.165542611579454,0.165925064968602,0.166293288298277,0.166628111072455,0.167028413566492,0.167479236630635,0.168015679718027,0.168876117330966,0.170125921835699,0.172043615127695,0.172820995234137,0.173786187940182,0.174754106470368,0.175914380083371,0.177302479255562,0.178730035268561,0.180150897577558,0.181963828403025,0.1838297971234,0.185941800709784,0.188333038123625,0.19057401186454,0.192063095397102,0.194790400852782,0.197327666013828,0.197263742036052,0.197007506775864,0.196724181670821,0.196415184781801,0.196075178478776,0.195702977199562,0.195656533965546,0.20000672387007,0.213995416573356,0.229317863196591,0.252837778313626,0.252347419101499,0.251598344005378,0.250785311842395,0.250084364546629,0.249477963769868,0.248835569973894,0.24813519371746,0.247552699492399,0.24679155017373,0.245956941431181,0.245038476504337,0.243995367736142,0.242915112022436,0.241437275527264,0.239651610736974,0.237629235746341,0.235431285658457,0.232996147227075,0.23034204025265,0.227425879314596,0.224203896969265,0.220579125443829,0.215665001787119,0.210271790287886,0.20435229229979,0.197854968037893,0.190700636941821,0.182828538744987,0.174127403967856,0.164043023655292,0.15296862668949,0.140817042715543,0.126659719643273,0.110243130148551,0.0862942321924339,0.0553143184139972,0.0213131774002569,0,0,0,0,0,0,0,0,0,0,0],[0.538303297876028,0.537887529260133,0.537593090746546,0.537437633060751,0.537468575337204,0.537580941443163,0.537698160830342,0.537820891295527,0.537906666052118,0.53791512143612,0.537877943376674,0.537826728122848,0.537750068696727,0.537652397682873,0.53754505750079,0.537434791872761,0.537319345072205,0.537179151500232,0.537033740609571,0.536877924667186,0.536708085736923,0.536636548169586,0.536676407264448,0.536739211687464,0.536834702818962,0.53713016002174,0.537462943776006,0.537836668302017,0.538238992169734,0.53868409830573,0.539177235738572,0.539377005537584,0.525584625210618,0.480527009750028,0.431169165305094,0.355853643326215,0.3540024410582,0.352823740300893,0.351686077418305,0.351152569180709,0.351049937457108,0.350970539781163,0.350767109242721,0.350352729204498,0.349711709464469,0.348941653922529,0.348098703964644,0.347225308247574,0.346295405795904,0.345195583705457,0.344037557379048,0.342740704676132,0.34132901892977,0.339767323459138,0.338063252550923,0.336191232499601,0.334121149259349,0.331833182280954,0.329004741065638,0.325900546885171,0.322493649850325,0.31875478682295,0.314637660759305,0.310077603411415,0.305183425299477,0.299717922287737,0.293716366101325,0.287130709886529,0.27943269623093,0.27049054876576,0.255908460391062,0.235833232749057,0.213798844649652,0.18978376371537,0.163666481122321,0.132541792853664,0.0932004401167079,0.0500233530807985,0.00263437213025965,0,0,0,0,0],[0.00100937495232314,0.00104128409942941,0.0010670157729008,0.00108539289707911,0.0011149137456659,0.00114528721417908,0.00117710339193056,0.00122515385652722,0.0012564267614142,0.00126845990596048,0.00119257160668585,0.00110480472832985,0.00099347917065443,0.000866012851177475,0.000728959849193397,0.000579002260153605,0.000413942974222783,0.000232848470134812,3.37339473960436e-05,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-0.0298592645375477,-0.0297811999271761,-0.0296922481665687,-0.0295924329604479,-0.0294725195323755,-0.0293373275572482,-0.0291877829674693,-0.0290108100204848,-0.0288125905946854,-0.0285787190603212,-0.0283812134942464,-0.0281632676009964,-0.0279287286883911,-0.0276581926879152,-0.0273458395956695,-0.0270058445272919,-0.026636237620653,-0.0262239773740936,-0.0257749742486292,-0.0252978030570576,-0.0247761657699034,-0.0241566993268072,-0.0235541115237425,-0.0228534806032789,-0.0221061858257678,-0.021356158814993,-0.0205398526523285,-0.0196426884624029,-0.0186592243899225,-0.0175793425397356,-0.0163940108888174,-0.0151779888778289,-0.0140548586241187,-0.0136857392458253,-0.0132784725882655,-0.0133083328817128,-0.0114517532998851,-0.00939504834152697,-0.00715379936563839,-0.00471835218867339,-0.00205096624941615,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.0442533647609414,0.0440976270680396,0.043939089596243,0.0437792072930732,0.04358259777686,0.0433664040987108,0.0431291424888704,0.0428427466019609,0.0425423093574827,0.0422174706889734,0.0420125029467458,0.0417879567043613,0.0415626595383215,0.0413279333488892,0.0410733358773728,0.0407969906231582,0.0404997005820629,0.0401639031971112,0.0397997528899246,0.0393807887595183,0.0389149555840609,0.0384188502876177,0.0378289560008276,0.0370445519103349,0.0362158821817915,0.0354022173694551,0.0345209318175099,0.0335484887657682,0.0324860643651297,0.031317864810031,0.0300341480563028,0.0288604504190377,0.0268412307019614,0.0243813562761098,0.0216843387100093,0.0181238022095095,0.0165685623550967,0.0148857461650401,0.0130307030453466,0.010973502599731,0.00871535761591111,0.00625795458764194,0.00367515685537701,0.0010582339267517,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.0341213444882743,0.0339746631706892,0.0338237692356691,0.0336694469120515,0.033488235292792,0.0332921208699395,0.0330788170242974,0.0328270180479398,0.0325804154494481,0.0323366976550913,0.032180351921677,0.0320162729041868,0.0318542189016602,0.0316821999349473,0.0314891966733994,0.0312765616210197,0.0310424272974962,0.0307871185441946,0.0305059900752568,0.0302059202928798,0.0298786050631873,0.0294565343469371,0.0289614512693329,0.0283193164635864,0.0276338497601133,0.0268424870490903,0.0259796412363676,0.0250260033286547,0.023985541309252,0.0228408454264247,0.0215824866522697,0.0205757436375669,0.0188981515548114,0.0173061965524458,0.015560172016809,0.0133826046057294,0.0119937640034693,0.010491376734513,0.0088334579735309,0.00700311289584762,0.00500899318883722,0.00281948589960946,0.000346326688275582,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-0.01825540904367,-0.0182553228712291,-0.0182439246429016,-0.0182170399674185,-0.0181918790208995,-0.0181661653921792,-0.0181441530869686,-0.0181539932201737,-0.0182093779504349,-0.0183495360867052,-0.0183539071634743,-0.0183748826120188,-0.0183897697866022,-0.0184182211929325,-0.0184614852470955,-0.0184963530064921,-0.0185160553861615,-0.018570298739836,-0.0186139157633697,-0.0186632744008388,-0.0187250389031356,-0.0187196235337799,-0.0185880412790789,-0.0185742712567798,-0.0184989624513948,-0.0180356243047762,-0.0175036418712153,-0.0169181337225049,-0.0162770704669229,-0.0155728097086627,-0.0147989426843093,-0.0140849034259117,-0.0143850979352251,-0.0170236416968551,-0.0199120341672261,-0.0247743619745901,-0.0236661889859817,-0.022380809256971,-0.0209501197633644,-0.019315299289914,-0.0174716618153708,-0.0153652861895698,-0.0128221828387765,-0.00990240486594162,-0.00684694279323485,-0.00360517010373352,-4.85917028329872e-05,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.0137721842985146,0.0137806399155615,0.013773566731322,0.0137491485099633,0.0137054465285534,0.01364855121729,0.0135854567712375,0.013520755707416,0.0134418944215871,0.0133480522030107,0.0132135733867202,0.013063297248296,0.0128938278598051,0.0126975889594408,0.0124739335649442,0.0122284018564068,0.0119590900621031,0.0116634578929928,0.011339002681574,0.0109845694472382,0.010596310111606,0.0101492471750291,0.00970372479385265,0.00925984286614081,0.00876824949386012,0.00825180287461979,0.00768520294366475,0.00706512857158604,0.00638296488966527,0.00563503158452376,0.00481495737675355,0.00379443762406827,0.00397217399112833,0.00635600853691979,0.00896469581721259,0.0135315541325718,0.0119480502686516,0.010152094673021,0.00817133682734192,0.00598486842935993,0.00359391412902053,0.00105816551009991,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.059207248539715,0.0591778941015799,0.0591569390906952,0.0591461920951202,0.0591629166878283,0.0591855206807015,0.0592037238957943,0.0592143538414743,0.0591607932446418,0.0590141394040908,0.0588698186322774,0.058691247022634,0.0584849287523281,0.0582359130849978,0.0579489744211213,0.05764498971206,0.0573275502030087,0.0569507448606729,0.0565509737900482,0.0561074264133683,0.0556126040132129,0.0551504399037698,0.0548095586272255,0.0544346990498668,0.054051351946434,0.0539795928229135,0.0539153335865033,0.0538521602008691,0.0537760396672033,0.0536955745848513,0.0536094483734971,0.0530800021880817,0.0512165322729976,0.0450292430726881,0.0382499018626449,0.0282813500255979,0.0271003072752396,0.0258933331457146,0.0245850304348286,0.023181363647104,0.0216432735701764,0.0199722349767811,0.0182521630690796,0.0163220478451587,0.0141506561323286,0.0117235102376997,0.00906397956165653,0.00592126782022063,0.00244988926076492,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[-1.17448480384475,-1.17444966736717,-1.1744163037479,-1.17438315811839,-1.17434698426228,-1.17430842577203,-1.17427014410193,-1.17425897868027,-1.17427541523018,-1.17435223924198,-1.17428498205399,-1.17421922859794,-1.17413799185116,-1.17406541611675,-1.17400446501952,-1.17392733006902,-1.17382863678313,-1.17374580134449,-1.17364245999515,-1.17352159362924,-1.17339486114947,-1.17325468773769,-1.17296247311664,-1.17276735007509,-1.17250108740848,-1.17194796332749,-1.17132219471539,-1.17063613432702,-1.16988251850738,-1.16905572809605,-1.1681489355579,-1.16704996079321,-1.16540580552134,-1.16284430761164,-1.16003594445161,-1.15629282100317,-1.15468699597442,-1.15294962604744,-1.15105436765096,-1.14900888200833,-1.14680040590885,-1.14438984337559,-1.14176701245449,-1.1387818612401,-1.1355159224813,-1.13194180550133,-1.12801673110168,-1.12388932001708,-1.11936605514793,-1.11432020970005,-1.10862885115576,-1.10237699900436,-1.09551657545112,-1.08798628518582,-1.07972273024066,-1.07065336860509,-1.06070252307913,-1.04995531859967,-1.03966783008543,-1.0283773248403,-1.01598587772973,-1.00238589651462,-0.987460767459809,-0.971099401386821,-0.95289159031968,-0.932303016562115,-0.909706250822665,-0.884906860267735,-0.857403797192708,-0.826888324975906,-0.793799599355057,-0.758437317624962,-0.719627840152231,-0.675981223247674,-0.626548082651789,-0.572057621921533,-0.511773787581314,-0.445612358329016,-0.373006393025898,-0.290836478096062,-0.199552210868363,-0.0983889084558275,0,0],[-0.223493815789978,-0.238142334425662,-0.252813925498383,-0.267279421512358,-0.285866581285104,-0.306122703386215,-0.328101423191894,-0.355568896228032,-0.381202728465774,-0.405805414140097,-0.4127376623263,-0.41922425013339,-0.423300040913852,-0.425050600332991,-0.425764055614245,-0.426512241895309,-0.427280960670557,-0.428216030393434,-0.429197442678735,-0.430299697150168,-0.431536656712177,-0.433614177453755,-0.448405585673996,-0.485552729344711,-0.522494528471996,-0.567827478432908,-0.616851551741469,-0.671698215954145,-0.730929531910887,-0.796372318967322,-0.868511041192939,-0.889437176342706,-0.983422212649356,-0.99855622295913,-1.01512887764642,-1.04182365775326,-1.03900915788761,-1.03545556581352,-1.03164968522041,-1.02695658680276,-1.02127863898871,-1.01511210295143,-1.00823814469201,-1.00041907083486,-0.99254833731375,-0.983966776731155,-0.974556160618555,-0.963603377536333,-0.951283456505606,-0.938480995215425,-0.923841571204763,-0.908097111581134,-0.890708650979185,-0.871740230088982,-0.850828959171639,-0.827895074400599,-0.80283438365,-0.775141013861019,-0.744194181973727,-0.710229909000115,-0.672956553150633,-0.632052910081856,-0.587275178286962,-0.538163191117309,-0.481632998791302,-0.412908980242916,-0.337507695060717,-0.254741416687171,-0.162983929121869,-0.0610038069705515,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.20269596441728,0.188005888407948,0.173254620105797,0.158655544943489,0.139923039591633,0.119505238342965,0.0973683933527158,0.0698753747069977,0.0443327168852561,0.0200938403000326,0.0128726044201017,0.00610717529781316,0.00168020478019281,0,0,0,0,0,0,0,0,0,-0.0124783647445984,-0.0488780871813933,-0.0852454637218387,-0.131099254794321,-0.180769549060174,-0.236323653643015,-0.296333037236587,-0.362629004462953,-0.435704301048131,-0.457880741794925,-0.550587292207917,-0.558626454568575,-0.567429260337815,-0.581754026100614,-0.579909854155436,-0.577577205004022,-0.575137569321298,-0.572212169960337,-0.568610791589861,-0.564936376870058,-0.561528855920364,-0.557683197901897,-0.553829688785631,-0.549472550019552,-0.54472206284946,-0.539257620303839,-0.532956434676108,-0.52667257795721,-0.519005737287138,-0.510935664843234,-0.501970142645826,-0.492245290674889,-0.481478344127888,-0.469677776307812,-0.456809989914671,-0.442481960581105,-0.425571847099496,-0.407012851549448,-0.3866474761171,-0.364302383425289,-0.339888122098189,-0.313019876283488,-0.281112695069325,-0.239196401680474,-0.193217223916989,-0.142742031485525,-0.0858510955491671,-0.0215139733283558,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.763632633136574,0.74922301091943,0.734737770886016,0.720373711703476,0.701943834458758,0.681868704478796,0.660132213669567,0.633256229453258,0.608540136280747,0.585519788272099,0.578799166299822,0.57264182221153,0.568818674532959,0.567498423770446,0.56749748925017,0.567434039494012,0.567278074342406,0.567262545923612,0.567167978746543,0.567058847053046,0.566971342171409,0.565657479877377,0.551428784694436,0.515446936362242,0.479242621080258,0.43202951457787,0.380770227375661,0.323475102026979,0.261552277281587,0.193157520961515,0.11777811415308,0.0931083734341465,0.00112130229548447,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.983893828153522,0.969691756034755,0.955393698850487,0.941182885818418,0.922943396257151,0.903079877846346,0.881598158574092,0.855170261493248,0.831097545634296,0.80908225493498,0.802635443041706,0.796834300187653,0.793340407110358,0.79248302328976,0.793127740126569,0.79372007601039,0.794204456982246,0.795030312499856,0.795792303584151,0.796640085589005,0.797641750058424,0.797447087275848,0.783982156940227,0.749243047432511,0.714125372029919,0.66656865926409,0.61482885647457,0.557009641648429,0.494508606934997,0.425480636988809,0.349407917511302,0.323871542932576,0.234242219586362,0.241601798497275,0.250319484659698,0.264545943279723,0.26197852237297,0.259039302153526,0.255742542951607,0.252177618048489,0.248409661293577,0.244299448543648,0.240180951026829,0.23635879546327,0.232082484828001,0.227449249725174,0.222349763585136,0.217080944023989,0.211427279638172,0.204844476288604,0.197715622893833,0.189745247619071,0.18104491298726,0.171446501031072,0.160953112141904,0.149429670531098,0.136747523495882,0.123014826091421,0.109280259225627,0.0942066610203742,0.0776620274531011,0.0595016503749529,0.0395239238550504,0.0176417534189441,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[1.72996231777587,1.71581838712914,1.7015525002746,1.68733408406233,1.66908689900906,1.64921746915736,1.62774953773863,1.60145933724142,1.57767534671178,1.55624902150395,1.54991785434255,1.54428777831261,1.54095603656337,1.54032960570007,1.54128350731812,1.54217274393597,1.54292546914963,1.54415032667404,1.54529825794452,1.54654139911879,1.54799502505741,1.54810926460547,1.53453763268885,1.50012184148786,1.46514137776901,1.4165688743894,1.36363783442968,1.30450787308942,1.24057139925573,1.16996662743745,1.0921606289451,1.06483609220928,0.976386060080981,0.991766403349774,1.00928860059654,1.0379068220217,1.03334560334307,1.02803004130939,1.02203768347955,1.01514482386881,1.00751041973035,0.998503156940962,0.986909562003465,0.974906057012818,0.962202948801152,0.948626671613333,0.933698566445973,0.918444693430287,0.901853799504538,0.883318274940745,0.863235920833987,0.841027735090256,0.816709862872251,0.789962455685437,0.76065590513944,0.728483770372054,0.693148233470821,0.654534297250216,0.613581921456184,0.568636827298122,0.519307619932097,0.465164595408519,0.405689422723378,0.34051803168799,0.271409416463712,0.203215194147863,0.128362110832008,0.0462180971589905,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[3.18781613609246,3.17273017147445,3.1574121966272,3.14200900572515,3.12243441540262,3.10111128655973,3.07808537356594,3.05028312314696,3.02514527534724,3.00274254647642,2.99439963658593,2.98666238820178,2.98097445070763,2.9778466529565,2.9761355752549,2.97401734811992,2.97134710121385,2.96903029793866,2.9661892708388,2.96291885540536,2.95945292889174,2.95375528646348,2.933001126338,2.89156067056318,2.84850802597781,2.7887370374095,2.72337633930277,2.65059764354721,2.57168835315207,2.48464792129266,2.38880443097525,2.34216310608614,2.23489521081082,2.23695395744578,2.23983723168736,2.25681198881763,2.22179625804419,2.18290658052674,2.14013325719177,2.09321050589898,2.04197181005673,1.98642855118391,1.92525857285844,1.85952149329685,1.78905727144477,1.71268114250312,1.62881448714703,1.53819443330753,1.4389708087656,1.32962639563089,1.2101413036722,1.0787355341545,0.93459573707264,0.776320550326831,0.602681491767548,0.412101516977163,0.20290359339561,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]],"plugins":{"Unzoom":"{}"}},"evals":["attrs.interactionModel","plugins.Unzoom"],"jsHooks":[]}</script> --- ## Elastic Net in R ```r ggplot(varImp(enfit, scale = FALSE)) ``` <img src="data:image/png;base64,#Day_3A_Slides_files/figure-html/unnamed-chunk-44-1.png" width="100%" /> --- ## Comprehension check .pull-left[ ### Question 1 **Which model uses an `\(L_2\)` squared penalty?** a) Ridge b) Lasso c) Elastic Net d) None of the above ] .pull-right[ ### Question 2 **What do the `\(\lambda\)` and `\(\alpha\)` hyperparameters refer to?** a) `\(\lambda\)` = shrinkage, `\(\alpha\)` = validation b) `\(\lambda\)` = feature selection, `\(\alpha\)` = mixing c) `\(\lambda\)` = shrinkage, `\(\alpha\)` = mixing d) `\(\lambda\)` = feature selection, `\(\alpha\)` = shrinkage ] --- class: inverse, center, middle # Live Coding --- class:onecol ## Live Coding Activity **Live Coding**: I will walk through examples of ridge, lasso, and elastic net for classification in RStudio. .footnote[ All files are on the course website: https://pittmethods.github.io/appliedml/. ] -- </br> **Small Group Activity**: Afterwards, we will split you into small breakout room groups to practice a full machine learning workflow using a new dataset. If you have any questions, please post them in the chat or workshop Slack channel. We will also float between different breakout rooms to answer questions.